{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "a06626b1-8b12-4482-87be-149128f7fb52",
   "metadata": {},
   "source": [
    "# ERNIE DPO Best Practices"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "426c7c9d",
   "metadata": {},
   "source": [
    "# 1 Introduction to DPO  \n",
    "### 1.1 What is DPO\n",
    "DPO (Direct Preference Optimization) is a method to optimize language models using human preference data, aiming to align model outputs more closely with human expectations. It bypasses the reward model training and reinforcement learning steps found in traditional RLHF (Reinforcement Learning from Human Feedback). The training data must include both positive examples (desired outputs) and negative examples (undesired outputs).\n",
    "\n",
    "### 1.2 Applicability of DPO\n",
    "- ✅ Applicable\n",
    "  - Behavioral alignment, aligning with human preferences and values. For example, making responses more friendly, polite, or safe;\n",
    "  - For open-ended questions with non-unique answers (such as content creation, summarization, or dialogue), enabling the model to learn more nuanced and popular ways of expression;\n",
    "  - Enforcing specific constraints or formats. For instance, encouraging the model to provide shorter responses.\n",
    "\n",
    "- ❌ Not Applicable\n",
    "  - Adding new internalized knowledge to the model;\n",
    "  - Forcing the model to acquire capabilities it does not possess;\n",
    "  - Fully correcting inherent biases in pre-trained models;\n",
    "  - Tasks with a single correct answer (such as strict mathematical calculations).\n",
    "\n",
    "### 1.3 General Steps for DPO\n",
    "1. Construct evaluation data and criteria, test the model's baseline capabilities, and analyze its shortcomings and issues.\n",
    "2. Conduct thorough prompt engineering.\n",
    "3. Determine if DPO is applicable; prepare training data.\n",
    "4. Train the model. Design training parameters based on task characteristics and data conditions. Key parameters to focus on include:\n",
    "    - Batch size\n",
    "    - Number of training epochs or maximum training steps\n",
    "    - Learning rate type and initial value\n",
    "    - Warm-up steps\n",
    "5. Evaluate the results. Use the same evaluation data and criteria to assess the model's performance after training. If the results are unsatisfactory, consider further optimizing the training data or adjusting the training parameters."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "9bd37505",
   "metadata": {},
   "source": [
    "# 2 Best Practice Task Introduction\n",
    "### 2.1 Task Description\n",
    "We have observed that in some scenarios, the model's responses are quite lengthy. We aim to make the model's responses shorter and more concise.\n",
    "\n",
    "### 2.2 Data Source\n",
    "We use the DPO demo dataset provided by the popular open-source community project LlamaFactory ([link](https://huggingface.co/datasets/llamafactory/DPO-En-Zh-20k/tree/main)). This dataset contains 1,000 entries each in Chinese and English, formatted in ShareGPT style, with the following fields:\n",
    "- `conversations`: Dialogue context, in the format `[{\"from\": \"\", \"value\": \"\"}]`, where the `from` field includes:\n",
    "  - `system`: System prompt\n",
    "  - `human`: User input\n",
    "  - `gpt`: Model input (here, `gpt` is used to represent the large model dialogue assistant)\n",
    "- `chosen`: Positive example (desired output)\n",
    "- `rejected`: Negative example (undesired output)\n",
    "> In real-world scenarios, we typically construct better data based on the model's current output to form training samples. However, for the purpose of demonstrating the DPO process in this tutorial, we directly downloaded the open-source dataset.\n",
    "\n",
    "### 2.3 Evaluation Criteria\n",
    "We focus on the length of the output, so we can evaluate the model's performance using the mean and median of the string length.\n",
    "\n",
    "### 2.4 Experimental Environment\n",
    "This tutorial uses the following environment:\n",
    "- 1x 80GB A800 GPU\n",
    "- CUDA Version: 12.3\n",
    "- CUDA Driver: 525.125.06\n",
    "- nvcc: 12.3\n",
    "- gcc: 12.2\n",
    "- Python Version: 3.10.12\n",
    "\n",
    "### 2.5 Dependencies\n",
    "- **ERNIEKit**: A toolchain for ERNIE large models, covering the full process of training, compression, and inference for the ERNIE 4.5 series models. Based on PaddlePaddle framework v3.1, it supports training on various mainstream domestic chips.\n",
    "- **ERNIEKit WebUI**: A visual interface supporting training, dialogue interaction, performance evaluation, model export, and more. [Documentation](../../docs/cli_webui_usage.md)\n",
    "- **[Optional] visualdl**: A tool for visualizing loss and other metrics, already included in ERNIEKit.\n",
    "- **ERNIEKit inference scripts**.\n",
    "- **Python dependencies** used in this tutorial."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "91ab45af",
   "metadata": {},
   "outputs": [],
   "source": [
    "# The Python dependencies\n",
    "import json\n",
    "import random\n",
    "from collections import defaultdict\n",
    "\n",
    "import matplotlib.pyplot as plt\n",
    "import pandas as pd\n",
    "import seaborn as sns\n",
    "\n",
    "RANDOM_SEED = 2025"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6a6d8592",
   "metadata": {},
   "source": [
    "# 3 Data Preprocessing"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "0612a29c",
   "metadata": {},
   "source": [
    "### 3.1 Preparing Raw Data\n",
    "First, download the open-source dataset and save it to `cookbook/data/llamafactory_dpo_en_10k.json`.  \n",
    "\n",
    "![dpo_download_dataset_en](https://raw.githubusercontent.com/wiki/Minghao2812/ERNIE/img/dpo_download_dataset_en.png)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "9752ae46",
   "metadata": {},
   "source": [
    "To facilitate processing, we convert the ShareGPT format to the ERNIE DPO format here. We have defined several data preprocessing functions for this purpose."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "ea18004a",
   "metadata": {},
   "outputs": [],
   "source": [
    "def conversations_to_ernie_data(conversations):\n",
    "    system = \"\"\n",
    "    src = []\n",
    "    tgt = []\n",
    "    for conversation in conversations:\n",
    "        if conversation.get(\"from\", \"\") == \"system\":\n",
    "            system = conversation.get(\"value\", \"\")\n",
    "        elif conversation.get(\"from\", \"\") == \"human\":\n",
    "            src.append(conversation.get(\"value\", \"\"))\n",
    "        elif conversation.get(\"from\", \"\") == \"gpt\":\n",
    "            tgt.append(conversation.get(\"value\", \"\"))\n",
    "    return system, src, tgt\n",
    "\n",
    "\n",
    "def is_valid_ernie_data(data):\n",
    "    if len(data[\"src\"]) == 0:\n",
    "        return False\n",
    "    if len(data[\"response\"][0]) == 0 or len(data[\"response\"][1]) == 0:\n",
    "        return False\n",
    "    return True"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6be94810",
   "metadata": {},
   "source": [
    "Call the above functions to process the data."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "94660786",
   "metadata": {},
   "outputs": [],
   "source": [
    "raw_data_path = \"../data/llamafactory_dpo_en_10k.json\"\n",
    "ernie_data_path = raw_data_path.replace(\".json\", \"_ernie_format.jsonl\")\n",
    "\n",
    "with open(raw_data_path, \"r\") as fin:\n",
    "    with open(ernie_data_path, \"w\") as fout:\n",
    "        data_list = json.load(fin)\n",
    "        for data in data_list:\n",
    "            system, src, tgt = conversations_to_ernie_data(data.get(\"conversations\", []))\n",
    "            chosen = data.get(\"chosen\", {}).get(\"value\", \"\")\n",
    "            rejected = data.get(\"rejected\", {}).get(\"value\", \"\")\n",
    "            ernie_data = {\n",
    "                \"system\": system,\n",
    "                \"src\": src,\n",
    "                \"tgt\": tgt,\n",
    "                \"response\": [[chosen], [rejected]],\n",
    "                \"sort\": [1, 0],\n",
    "            }  # The smaller value in 'sort' represents 'rejected', while the larger value represents 'chosen'\n",
    "            if is_valid_ernie_data(ernie_data):\n",
    "                fout.write(json.dumps(ernie_data, ensure_ascii=False) + \"\\n\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "03201de0",
   "metadata": {},
   "source": [
    "### 3.2 Analyzing Raw Data\n",
    "Observe the length distribution of `chosen` and `rejected` in the dataset, then construct the test set and training set accordingly."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "62eedc9a",
   "metadata": {},
   "outputs": [],
   "source": [
    "chosen_len_cnt = defaultdict(int)\n",
    "rejected_len_cnt = defaultdict(int)\n",
    "\n",
    "with open(ernie_data_path, \"r\") as fin:\n",
    "    for line in fin:\n",
    "        data = json.loads(line)\n",
    "        chosen_len = len(data[\"response\"][0][0])\n",
    "        rejected_len = len(data[\"response\"][1][0])\n",
    "        chosen_len_cnt[chosen_len] = chosen_len_cnt.get(chosen_len, 0) + 1\n",
    "        rejected_len_cnt[rejected_len] = rejected_len_cnt.get(rejected_len, 0) + 1"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8f0719d6",
   "metadata": {},
   "source": [
    "Perform a simple visualization analysis."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "944b5348",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAABdIAAAJOCAYAAACz9fURAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjkuNCwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8ekN5oAAAACXBIWXMAAA9hAAAPYQGoP6dpAACnpUlEQVR4nOzdeVxUZf//8fewCq7hgoIoirkrotyamaVmqWVlmFullXWruXR3W2mKabnk1mImmZqlZrnvS1qWS5YtNypWZrmDCgriUokCw/z+4Mv5MYLjDA4Mo6/n48FDmXOdOZ9rPuN4nc9c5zomi8ViEQAAAAAAAAAAyJeHqwMAAAAAAAAAAKA4o5AOAAAAAAAAAIANFNIBAAAAAAAAALCBQjoAAAAAAAAAADZQSAcAAAAAAAAAwAYK6QAAAAAAAAAA2EAhHQAAAAAAAAAAGyikAwAAAAAAAABgA4V0AAAAAAAAAABs8HJ1AMDN5NVXX9WqVausHjOZTPLz81OVKlV0991365lnnlFgYOA19/v6669VtWrVAh0/KytLBw8eVJ06dexqf+LECd17772SpEcffVSTJk2SJPXu3Vs//fSTJOmPP/4oUCz2SEhI0G233aZSpUpJkn788Uf16dNHkjR48GANGTKk0I5dGH766Se99dZb+uOPP+Tp6ak6dero888/l8lkuuY+ly9f1qJFi7R582YdPnxYaWlpqlChgpo1a6Y+ffooPDzcqn1R5aY4ateunU6ePJnncU9PT/n7+6tatWrq2LGjnn76afn4+LggwltHzmdM8+bN9emnn7o4Gsf9/fffOnfunEJCQozH3L1PAHCrYdztmFt53J37Nc7Nw8NDfn5+CgoKUrt27dSvXz/j9XFUzjg1ODhY33zzTYGew1FX59RZ7rvvPsXHx1+3L7nf01fz9vZW6dKlVatWLT3++OPq1KmTU2OENXf/9yzdfJ9RuHkxIx0oZBaLRZcuXdLhw4f1ySef6NFHH9WBAwecfpxvv/1WjzzyiD755BOnP7eznT9/XhMnTlSnTp10/vx5V4fjFGlpaXr++ecVFxeny5cv659//tFff/1ls4h+9OhRPfzww5o0aZL27NmjixcvKiMjQ4mJiVq/fr169uypuXPnFmEv3JPZbNZff/2l3377TW+//baGDh3q6pBQTGVmZuqzzz7Tfffdp59//tnV4QAAnIxxd16Mu68tKytL//zzjw4ePKhZs2apb9++ysjIKKSonccdcpqRkaHU1FT99NNPevHFF7VgwQJXh4Riyh3ez0BuzEgHCsmSJUtUuXJlZWRk6PTp01qzZo2WLl2qs2fPasiQIVq/fr18fX0lSSNGjNCLL74oSapYsaLDxzp58qSee+45SVKDBg3s3q9KlSravn27JMnPz8/h4xbU5MmTtXLlyjyPR0REGPE4e2ZFYTt06JD+/vtvSdJdd92l0aNHKysr65rtL168qOeee04nTpyQh4eHnn32WXXu3FkeHh7atm2bZsyYoStXrmjq1Klq2LChWrRoUVRdKfYqV66sJUuWGL9nZmbq8OHDGjlypFJSUvTVV19p9+7datq0qQujRHG0bt06jR071tVhAACcjHH3tTHutpbTZyl7MsapU6f02muv6ejRo4qLi9OmTZv00EMPORzTkiVLZDab5enp6fC+jrpWTl2pY8eOGjFihKTsL7QyMzO1Y8cOjR8/XllZWZo+fbq6detWpO99uIfi+H4GbKGQDhSSChUqqHLlypKkkJAQRUZGysPDQ4sXL1Z8fLzWrFmj7t27S5LKli2rsmXLFvhYFoulQPt5enoaMRala8Xr4+PjknicIS0tzfh7eHi4qlevbrP93LlzdeLECUnS8OHD9fTTTxvbateurdtuu02jRo2SxWLRypUrKaTnkt/7tmrVqnrqqaf09ttvS5L27dtHIR15FPSzEgBQvDHuvjbG3dau7nNwcLBeeOEF/fe//5UkxcXFFaiQXpAvZQqqOI5n/Pz88ry2TzzxhLZu3apvv/1Wf/31l44ePar69eu7KEIUV8Xx/QzYwtIuQBHq3bu38ffc6829+uqrqlOnjurUqWMUVyVpz549GjBggFq2bKn69euradOm6tatm1asWGG0WblypdXadKtWrVKdOnWMb3XbtWunOnXq6IUXXtBHH32kFi1aqEmTJpo2bZpOnDhhHPfVV1/NN+aTJ09qyJAhioiIUGRkpAYPHqyjR4/m6VfO8+T2448/Go+///77Rjy517O899571a5du2u2z/H333/rvffe04MPPqjw8HBFRkbqqaee0ldffZUn5tx9PnLkiAYOHKhmzZopIiJCAwYMyBP/tWRlZWnp0qXq2bOnIiMj1aRJE3Xp0kUff/yx0tPTrfqfO7cxMTFWOchPzmtQtmxZPfHEE3m2P/LII5o0aZK++uorTZ48Od/nSE1N1ciRI9WiRQtFRETo2Wefzffy5TNnzujNN9/U/fffr0aNGqlFixYaMGBAvmtFHj58WC+++KLuuusuNWjQQE2aNNEjjzyijz76KM9Mn6ysLC1YsEAPPfSQGjVqpObNm2vAgAHat2+fVbvced26datWr16thx9+WI0aNVKbNm307rvvOuUyWi+v///dcIkSJay27dixQ71791ZERIQiIiLUvXt3rV27tsD9z/1vZ968eVq3bp3Rp3vvvVcffvihzGZznuf/5Zdf9NJLL6l169Zq2LCh7rnnHkVHRyshIcGqXUFes23btumpp55S8+bNVb9+ff3rX//Sk08+qa+//jpPHBcvXtTEiRPVrl07NWzYUK1bt9Zrr72m06dP2/diO+jKlSuaMWOGOnTooIYNG+rOO+/U0KFDdeTIEat2K1euNPp94MABzZs3z9jn/vvvz/cS+tTUVI0ePVqtWrVSeHi4nnjiCe3du9f4XM35fHn11VeNmVJS9ozEqz9zcxw4cEB9+/ZVkyZNdMcddyg6OlqpqalWbez5fAYAuA7jbsbd9so9i/zqGdNxcXHq16+fIiMj1bhxYz3yyCP69NNP84yLc16HnNc3hyNjrosXL2rq1Knq0KGDGjVqpDvvvFNPPfWU1Sx6WzmV7B9zSdLp06f16quv6o477lB4eLiefvpppy+FdK3xub3nEZL9Y9zc48h9+/bp/fffV9u2bdWoUSNFRUXpyy+/zDfGTZs26emnn1aLFi3UqFEjPfDAA3rvvfeMqx5yvP/++8bznzt3Tu+9957atm2rhg0b6qGHHtLq1aut2mdlZWnevHl69NFHFRERoQYNGqhVq1YaMmSIDh48mCeOnPOQO+64Q40aNVLHjh0VExOjK1euXPd1LojTp08rOjpad911lxo2bKh27dpp0qRJunjxolW7nM/MFi1a6K+//tK4ceN01113qVGjRnrssce0Y8eOPM994MAB9e/fX02bNlWzZs00dOhQnTlzxvh3kvMZeL33c25ffPGFHnnkETVq1Ejt2rXTBx98kOd8a/Xq1erZs6eaNWum+vXrq0WLFnr22WcVGxt7oy8XYGBGOlCEwsLCVKJECV2+fFm///67zbb79u1T7969rYpl//zzj/bt26d9+/bp4sWLeuaZZ+w+9q5du7R582bj96tvYnktPXv21JkzZ4zfv/rqK/34449atmyZQkND7T7+jTh9+rQef/xxq5Ody5cv64cfftAPP/ygp556SiNHjsyz39GjR9W9e3f99ddfxmNbt27VwYMHtXnzZquB3dXMZrMGDBiQZ2Dw+++/6/fff9dXX32luXPnyt/f3+H+JCYmGoPnhg0bytvbO08bHx8fPfroozaf5/HHH7c6Odm5c6d+/fVXffPNNypZsqSk7EHM008/rXPnzhnt0tPTtXXrVm3btk2vvvqqMRv+5MmT6tGjh9XrlZmZqQMHDujAgQNKSkrSqFGjjG0vv/yyNmzYkOd5d+7cqRkzZqhNmzZ5Yv7oo4/0v//9z+q1+PDDD+Xh4aH//Oc/NvubH4vForS0NP3555/GDSI9PT31r3/9y2jz+eefa+zYsVazHeLi4hQXF6dDhw4Za6o72v8cK1eutLo52IkTJ/Tuu+/q4MGDxgx5KXtgFx0drczMTOOxpKQkLV++XJs2bdKsWbMUGRmZ5/ntec2++uorDRkyxKqPFy9e1M8//6z//e9/mjZtmjp27ChJunDhgnr27Gl1QnXmzBktXbpUW7du1ZIlSxQcHHzN19xR6enp6tu3r1Ufzp49qw0bNmjbtm1asGCBGjZsmGe/cePGWe1z/PhxTZo0SaVLl9Zjjz0mSfrrr7/Uq1cvHTt2zGj3v//9T3369FGNGjUKFO/x48fVq1cvXbp0SVL2rLfly5crKSnJuGeBsz+fAQDOx7i7YG62cbctly9f1vHjxzVz5kzjsdxXgX799df6z3/+Y/W+OHDggMaPH6+9e/dajfPy48iY69y5c+rZs6fVmObs2bM6e/asfvjhB40fP17dunWzeTxHxlypqanq0aOHEhMTjba7du3S448/bvMY9sjKytLff/+tHTt2aOfOnZKylzaqVq2a0cbe8whHxri5jRo1ymp8/ttvv2nIkCF644031LNnT+Px0aNHWy0ZKWUXtD/44ANt2rRJn376qSpUqJDn+QcPHmz1Ov/5558aPny4AgMD1bJlS0nSpEmTNH/+fKv9UlJS9OWXXxr/rnOupti3b5+efvpp/fPPP0bbo0ePavr06dq1a5c++eSTfM8ZCyohIUG9evVScnKy8djJkyf1ySefaMeOHVqyZIlKly5ttU9GRoZ69+5t9Xn6yy+/aMCAAdqwYYMx9v7ll1/Uu3dvqytHNmzYoF9++cUYXztqw4YNVue9J0+e1HvvvSdPT0/1799fkjR//ny9+eabVvudP39eO3fu1E8//aT58+dzxTKcghnpQBEymUzGf0gXLlyw2XbNmjXKyMiQv7+/PvroI23ZskWff/65br/9dnl5eWnTpk3KyspSp06drP7z79ixo7Zv357nzugXL15Uhw4d9MUXX+jdd99V69at7YrZ399fH3/8sdasWWMUdi9evKi33nrLka4blixZYjXYWbJkSZ7By9VGjhxpDOb79OmjNWvWaN68eca6lPPnz7caiOX4888/1bBhQy1fvlzLli1TYGCgpOxC5/VuNDh37lxjMN+yZUstXrxYK1euVOfOnSVJu3fv1qRJkyRJ7733nqZNm2bs+/TTT+ebgxy5Byy33XabzThs8ff31+eff65169YZJ2jnz583ZgtlZWVp6NChOnfunDw9PfWf//xHGzZs0MyZMxUSEiKLxaJJkyZp9+7dkqTNmzcbJz/Tpk3Tli1btGLFCjVv3lweHh769ttvjZkZX3zxhfGad+nSRevWrdOSJUsUERGhjIwMjRw50mr2UI7Y2FgNHTpUmzdvVnR0tPG4I7OITp48acwGqVu3riIiItSjRw+dOnVKJpNJgwcPVlhYmKTsk8E333xTFotFjRo10ueff64NGzYYJyKzZ8/Wb7/95nD/c/vjjz8UFRWlNWvW6KOPPlLVqlUlSevXrzdOHhISEjR69GhlZmbqtttu09SpU7Vx40aNGTNG/v7++vvvvzVkyJB8n9+e12zFihWyWCyqUqWKPvvsM23ZskUff/yxKlWqJC8vL23cuNFoO23aNB05ckQmk0mvvvqqNm3apJkzZ6pixYpKTk7W+PHj7c6FPRYsWGCcaDz33HPauHGj5s+fr5o1a+qff/6x6lNu+/bt07hx4/Tll1/q3//+d779njNnjnHC2apVKy1fvlyLFy9Wo0aN8syoGjFiRJ4Z6du3b1eVKlWs2p0+fVr33nuv1q9frwULFqhcuXKSsr+oyjnZtPfzGQDgOoy7GXfnJ2cMWadOHYWHh+vhhx82CoPdunXTXXfdJSn7i/RRo0YpIyND1apV09y5c/XFF19o4MCBkrLHefnN0M/NkTHXu+++a4xpnnjiCa1bt07z5883Cu2TJ0/W33//bTOnjoy5PvzwQ2Nc07p1a61YsUKff/65atSoYVXMtVfO1Rl16tRRvXr19K9//UsvvfSS8e9q3LhxxpcpjpxHODLGze3w4cMaPny4Nm7cqPHjxxuz4SdPnmx8HmzYsMF47erVq6d58+Zp7dq1xiSjI0eOaPjw4fk+/8GDBzVt2jRt3rxZjzzyiPF47nHqsmXLJGWv47969Wpt2bJF48ePl6enp8xms7Zs2SIpe2JQdHS0/vnnH912222aMWOGNm3apNdee00eHh76+eef9dlnnzmcE1vGjRun5ORk+fr6atKkSdq8ebPeeust+fn56fDhw3rvvffy7PPPP//o4sWLmjNnjjZu3KhWrVpJyv4ibM2aNUa7CRMmGEX0vn37av369YqJiVF6errV5C7J/s+oY8eO6ZVXXtGXX36psWPHGjcWznmNJWn58uWSpPr162vZsmXasmWLpk+frpIlS8rT01NffPFFQV8uwAqFdMBFcs9KzU9O8ebKlSv64YcfdPr0aTVs2FCLFi3Snj17tGTJEnl4eMjPz8/qW/Kc9enyu5HLsGHDVLNmTT3wwAM2Z4Xk9sYbb6hVq1aqW7euxo8fbwzmvv322+v2IT8VK1a0iq1ChQo21xQ8fvy4UYy8++67FR0drbp166ply5aaOXOmceOonNnIuZlMJr3zzjtq1KiRGjdurKeeesrYlruYnZ+cwUpAQIBiYmKMy/GmTp2qunXrSsoeKP3zzz8KCAhQQECAsW+pUqWumQNJVgW2G1kTbsyYMWrWrJlq165tVWjMme3+ww8/6PDhw5Kk7t27a+DAgapVq5batWtnnIBYLBajrznvOUn6/vvvFR8frxo1amjWrFnas2ePNm/ebNyMav369ZIkb29vDRkyRGXKlFHlypWNGQFnz57N9zK/Nm3aqH///goNDVWfPn10++23S8qenXEjPD091bVrV82fP984wZGyL9XMmUX073//W8HBwSpVqpQGDhwoPz8/WSwW4zJMR/qfW2hoqMaPH6+6deuqdevWeuONN4xtOQPk5cuXG5dljh49Wg8//LDCwsL0+OOPa8iQIZKyZwbld2Jqz2uWE/uFCxf0448/6vz582rRooXWr1+vuLg4TZ8+XVJ2vnNOOJo0aaJOnTrJz89P9evXV9euXSVlXz579SD3RuS8V4KCgtS7d2+VLFlSoaGhxmXZBw4cyHemYPfu3dW9e3dVr15dL730knGVRe5+51zS6+fnZ/xbj4iI0HvvvWd8NuQoW7asypQpY/ye8569+qZgpUuX1ptvvqnbb79dLVq0sFonNSkpSZL9n88AgOKBcXe2W3HcbY8OHTooJibGqrD93XffGcu6Pfnkk6pVq5b8/f3Vo0cPYxZx7iUprubImCsrK8so8oWGhuq1115T7dq1dccdd+jNN9/UqFGjFBMTI29vb5s5dWTMlbNcjL+/v9555x01bNhQzZo1u+4se0eUK1dOzz33nFavXm31ZZIj5xH2jnGv9sgjj6hv374KCwtTt27djPfjpUuX9N1330mSFi5caMQxc+ZMtWzZUnXq1NGIESPUtm1bSdkTKfJbnmjAgAHq1KmTQkNDrZZqyj1OzZkwFR8fr71798psNqtbt27auXOnYmNj9eyzz0rKnpTz559/SpIeffRRNWrUSH5+fmrfvr1xla2t95qjLly4oG+//VZS9lIqLVu2VIkSJfSvf/1L999/vyRp7dq1+Z6nDh8+XHfffbfCwsKM+wrk7ndqaqr27NkjKft9P3z4cN1+++1q3769Ro8enef57P2Muuuuu/Tcc8+pevXq6tGjh2rXri3p/4/Npf//Xjl9+rRiY2N16dIl3Xffffr666+1Z8+ea07eARzF0i5AEcuZcZq7aJef3r1765tvvtH+/fv10Ucf6aOPPpK3t7caNWqk++67T927d8+3qHct/v7+xkxZRzRu3Nj4u5eXlxo0aKCTJ0/q8uXLOnfunM3BuDNuHJJ7Vumdd95ptS0wMFBhYWHav3+/1aV7Oa4eaOf+u62TkXPnzhn/KYeHhxsFPEny8PDQHXfcoQMHDigjI0NHjhxRo0aNHOpT7jjOnj17zXZZWVk2i3G518bM/V7IKRzbeu0aNmyosmXL6sKFC8Zr17lzZ23YsEE7d+7U0qVLtXTpUnl6eqpu3bpq166devbsaZw85syYycjIsForNLdff/1V7du3t3osZ6Z4jpwBpiMnh5UrV9aiRYuUkpKiOXPm6Msvv5TZbNapU6fy3MAo9+WxL7zwQr7PlzMj3ZH+59aoUSOrYmzuSwZzZnTlfn9enYvcv+f3PrbnNRs0aJB+/vlnnThxQtOnT9f06dPl5+eniIgIdejQQVFRUfLx8dG5c+d0/vx5Sdlrwd5zzz15jpeVlaUDBw4Yl6XeqJwcnDp1Kt/jSdnvlXr16lk9lrvfJpNJ5cqV0z///GPV75y15WvUqGH1mVqhQgWFhobm+3peT40aNeTj42P8nvuy1px/W87+fAYAFA7G3Y65GcfdV9u2bZsuXLigRYsWafHixZKyC51XP2/uMeSbb76ZZ8kI6f+PIfPjyJirdu3axrrUderUMWbbStIdd9yhO+64w66+OTLmOnXqlKTscU/uiQahoaEqV66cEbu9OnbsqFdeeUXHjh3T5MmT9eeff+rChQvKysqyWtIld5z2nEfYO8a9WkREhNXv+Y3Pc97vNWrUyHOF4p133qmtW7dKyh6fX71kYO5xakBAgEwmkywWi9V7/bXXXtPQoUMVHx+v119/3WjbokULdenSxVi+Jneh/uOPP9bHH3+cpz8HDx5Uenp6vn111PHjx42JXRs3bsx3Vv+FCxeUkJCQJ3e1atUy/p7fv/Hc931q0qSJ1b75LWFpr6vvCZEzPs+97NIrr7yifv366ezZs8YVLKVLl1ZkZKQefPBBPfjgg0x0gVPwLgKKUHx8vHGZU87simspV66cli9frpkzZ6p79+4KDQ1VRkaGdu/ercmTJ6tr1655bgRiS0GLOlcPfHP/53P1TE5JVjf8cMaNUfI7Rm62ThquvuGkvf9xXm/WUO5j5h7o2iskJMQohv7222/5LoGSmpqqli1b6r///a8xq/lqufuXu2858V3vtcuR0wcfHx/NnTtX8+fPV+/evVW7dm1ZLBb99ttvev/999W5c2edPHnS7ue++uaMV8fsSIxX7xMUFKTGjRtr2rRpat68uaTsdR1feOEFqxn/jsTpSP9zuzp/+b0/HM1Fbva8ZiEhIfriiy/0zjvv6OGHH1ZQUJDS0tL0/fffa8yYMerTp48yMjLsjiO/3BWUPcfMbwa8Pf3OWSvSGcWDax03v39bzv58BgA4H+Nux92M4+6rValSRXXr1tUbb7xhLMnx+++/67nnnrNa09meqwhsjZccGXPl7mNBrjxw5Jg5Y66c1zK/nBak2Ojn56eqVavqrrvu0ty5c1W+fHlZLBZ9/PHHiomJcTjOnNfW3jHu1ewZn9vK8fXed/a839u2bauvv/5ar732mtq0aaOyZcsqNTVVX3zxhfr3768pU6ZcN44cZrP5uktU2cveK2Tye3/nvuLT1thcKrrxeY7GjRtry5YtmjBhgjp06KAKFSror7/+0tatW/Xyyy9bzaAHbgSFdKAI5V47LOeyqWs5evSotm/frrS0NI0bN06bN2/Wrl27jCU8jh07ZlySl/s/92v9h1XQm5PkXtPQbDZr//79krK/3c2Z3ZP7m/HcazznvnlNbvbEmyP3t/27du2y2nb69Gnj5j1Xz2a9EaVLlzZm/Ozbt89qncCsrCz9+OOPkrL7XbNmTYef32Qy6eGHH5aUfbPERYsW5Wkzb948nT9/Xhs3bjTy7Chbr92vv/5qDMZyTi5PnjypHTt2KCEhQaNGjdK6dev0888/G5fBnTt3TuvWrZMk45JWPz8//frrr/rjjz/0xx9/6Pvvv9fatWv1yy+/aOzYsQWK2xGenp6aPHmyccL6/fffa8GCBcb2nDgladGiRUacv/32m5YuXarY2Fht2rTJ4f7nFhcXZzVY/+WXX/Ic31Yuvv/+e+Pv1zvRz09WVpYOHjyobdu2yd/fX1OnTtXWrVu1Y8cOdenSRVL2TKhffvlFZcuWNf7dtmrVyng9/vjjD23evFlbtmzR77//rgcffNDhOK4l5zXImSGe87Nt2zZ98cUX+u2339SvX78CPXdISIik7M/D3DOnzpw5k+9luI589tjiyOczAMA1GHfL7nhz3IzjbltGjx6toKAgSdlrvOde1iT3TNy33nrLagyzbNky/fjjj1Zjvqs5MuYKCAgwZtgeOHDA6guSLVu26PHHH9eoUaOMJRuvlVNHxlw5/Tt69KjV++jYsWM3PKGiUqVKVucBH3zwQb7j4+udRzgyxr3aTz/9ZPW7rfH50aNHrZYIkazf/wV5v1++fFn79u3Tt99+q6ZNm2rWrFn68ccftWHDBmO8/9lnn8lsNlu91/773/9a5W7VqlXauXOn/vjjD5tXpDgi9/G6d+9udbz169dr27Zt+uOPP/LMKLdHzthckvbu3Wu1Leff8tWcMT7PyMjQ77//rh07dqhatWqaPn26vvvuO23ZssW478GmTZuMJVCBG8HSLkAhSUlJkZeXl7KysnTx4kV98803mjVrliSpatWqxn/+1/LGG29o165d8vDwUHJystq0aaPLly9bffjnfJuc+5vh48eP6/Dhw/L3989ziVpBvPbaazKbzapRo4YWLFig+Ph4SVL79u2Nb4IrVapktF+wYIEGDRqkP/74Q7Nnz873OXOfAOzbt0/nz5+/5mWaNWrUULNmzRQbG6vt27dr4sSJioqK0vnz5zVlyhRj9s2TTz55w33N7bHHHtPMmTN19uxZDRkyRC+88IJ8fHw0b9484zLArl27yt/fv0DPP2DAAOM/88mTJys1NVUPPvigsrKytG7dOuOSvhIlShS4yNiyZUsFBwfr5MmTWrJkiQIDA9W+fXudOHHCuDzVZDIZr90HH3xg3KQlPj7eeI/mPjHLec899NBD+vLLL5WWlqZhw4bp3//+tzw9PTVx4kTt2rVLnp6eWr58eZ6lVgpDUFCQXn31VY0aNUpS9o2d7rvvPgUHB+v+++/X1KlTdeXKFY0dO1avvvqqqlSpokWLFumTTz6RJEVHR6tPnz4O9T+3pKQkvfLKKxowYIAuXLhgXLopSffdd5+k7HUa586dq8zMTI0dO1Zms1n16tXTzz//rPfff19S9nIk9t4o62qDBw/WsWPHVKJECb3++utq1qyZzp8/b7VOY+7cffrpp/r+++/14Ycf6v7779fZs2f18ssvKykpSYGBgfryyy/zzPzIz/nz5/NdC1/Kno3XtGlTPfTQQ/rtt9907NgxjR8/Xj169NDly5cVHR2tP/74Q/7+/tq8ebPV54i9HnzwQf3+++9KS0vTSy+9pKFDhyotLU1Tp07N90qP3J+Vv//+u37//XdVr17d4X/Hjnw+AwAKH+Nuxt0FUapUKY0fP159+/aVlF3YfOihhxQeHq6WLVuqQoUKSklJ0bvvvqtSpUopLCxMW7Zs0eTJkyVJzzzzjNX62FdzZMzVsWNHLVu2TCdPntTo0aPVp08f/fXXX5o6daqOHTum/fv3Gze+vFZOHRlz3X///Tp48KDS0tL0wgsv6MUXX1RGRoYmTJjglNe2ffv2euihh7Ru3TqZzWaNGjVKK1askJeXl93nEXXr1nVojJvb5s2bFRMTo44dO+qXX37RvHnzJGXnPGfZoq5du2rPnj3KyMjQwIEDNWzYMAUEBGjVqlXGsi533313nuVN7JGSkqKePXvKbDarevXqGjNmjKpVq6bTp0/rr7/+kpQ9q9pkMql27dqqW7euDhw4oI8//lghISFq1KiR9uzZoxEjRshsNuv+++83zhmu5/jx49ccnwcHByssLExt27bV1q1btWrVKjVo0EB33HGHjh49qpdffll///23GjRooBUrVjh8FUjp0qV19913a8eOHYqLizOu6Dl8+LDGjRuX7z6OfEZdS3p6uvr06aOLFy8qICBA48aNU506dXT69GmrqzAKcjU0cDXO8oBC0qNHj3wfL1eunN5///3rrm82YsQIPfXUUzp37pwmTpyoiRMnWm2vX7++saZcQECAcff3PXv26IEHHtArr7yi55577ob6EBQUpIyMDONmiDkqVqxodWlU586djTuUz5gxQzExMbJYLGrSpEm+S2HknnU7dOhQ+fn55fnGOrfJkyfrySefVFJSkubNm2cMhHI89dRTeuCBBwrQw2sbOHCg9uzZox9++EHfffedcVOaHE2bNr3mXdztERAQoDlz5mjAgAE6deqUPvzwQ3344YdWbXx8fPTWW29ZfbPvCE9PT02bNk3PPfecLly4oHfffVfvvvuusd1kMunVV19VeHi4pOw1xH/88UclJCRo9uzZeU7IgoKCFBUVJSm7QHzPPfdo+/bt+a6t99hjjxVJET1Ht27dtGnTJu3cuVNpaWkaO3asZs2apcDAQA0ePFhvv/22fv/9d6sbX0lSgwYNjBs+OdL/3AIDA7V582ZjZnuOqKgotWjRQlL2jJdRo0Zp3LhxSk1N1UsvvWTVtlSpUnrvvfcKdCm4h4eHXn/9dQ0YMECXL1/O94SuTZs2xrqrAwYM0DfffKOTJ0/meU94eHjopZdesquILmXP3sp9o9vc6tatqzVr1qhXr15at26dfvvtN3366ad5blDWv3//AhXRJenxxx/XypUrdeTIEe3cudO4QVqZMmWMz8Tccq+vmBPLsmXLrNaktYcjn88AgMLHuJtxd0G1atVKPXr00JIlS5SVlaXRo0drxYoV8vPz04gRI/TKK6/o5MmTGjBggNV+wcHBeuaZZ2w+tyNjrqFDh+qnn37S8ePHtXz5cmNyR47o6Ghj1vq1curImOvZZ5/VV199pT///NPqNffz81ONGjXyvbLPUaNGjdIPP/yg5ORkHThwQPPnz9ezzz7r0HmEI2Pc3CpXrmysqZ7DZDIpOjraGG8/9thj+umnn7R27Vr99ttvec4TatasmeezwF5Vq1bVf/7zH73zzjs6fvy48WVNboMGDTK+IIuOjjbO2YYOHWrVrly5cho8eLDdx163bl2+V9FKUp8+fRQdHa1XXnlFe/bs0fnz5zVmzBirNiVKlNArr7xS4KWUXnrpJf38889KS0uzWvM9LCzMGJvnfm5HP6PyU7JkSb322msaPny4UlNTNWjQoDxtevXqle/9rgBHsbQLUAT8/Px0++2369lnn9X69evtKjDWqVNHy5cv15NPPqnQ0FD5+fnJ19dXtWrV0oABA/Tpp58aJwUeHh4aO3as6tSpI19fX1WsWNHqpjEFddttt2nJkiXq0KGDSpUqpdKlS+uBBx7Q0qVLFRgYaLRr1aqVpk6dqtq1a8vHx0dBQUEaNGiQZs6cme/zPvLII3rooYdUvnx5+fn5qVatWrp8+fI14wgJCdH69es1cOBA3X777SpRooRKliypFi1aaMaMGRo5cuQN9/VqPj4++uSTTzRu3Dg1a9ZMpUuXlq+vr+rWravhw4dr/vz5VncYL4g6depo3bp1+u9//6sGDRrI399f3t7eqlq1qrp37661a9caM5oLqnHjxlq/fr2eeuophYaGysfHR2XLllWbNm00f/58Pf3000bbwMBALV26VP3791etWrVUsmRJeXt7q1q1anryySe1bNky46YyJpNJM2bM0PDhw1WvXj35+fmpVKlSatiwocaNG6c33njjhuIuiPHjxxsnGNu2bTMK2/369VNMTIxatGihMmXKyNfXV6GhoRowYIAWLFhg3NTKkf7nduedd2rGjBmqXbu2vL29FRISopdeeinPjJ5evXppyZIl6ty5sypVqiRvb28FBgaqa9euWr169Q3dgKdly5ZatmyZunTpopCQEPn4+MjPz0/16tXTyy+/bDWDpUKFClq2bJn69OmjkJAQeXt7KyAgQHfddZc++eQTY71QZylRooQWLFiggQMHKiwsTL6+vipbtqwiIyM1ffr0PCemjihZsqQWLlyoqKgolStXTiVKlFCrVq302WefGSeKuYsnYWFh+s9//qOqVavKx8dHoaGhBTquI5/PAICixbjbGuPu6xs2bJiCg4MlySj4StlfWsyfP19t2rRRuXLljHF67969jas9bXFkzBUQEKBly5apb9++qlatmry9vVWhQgW1atVKc+fOVbdu3Yy218qpI2OukiVL6tNPP1WPHj102223yc/PzxhD1a5d2ymva7ly5azOCWbMmKGTJ086dB7hyBg3t8GDB+u///2vKleuLB8fHzVo0EAxMTFWk2JMJpOmTp2qadOmqVWrVkaOa9SooYEDB2rZsmU3VHjt37+/Zs+erdatWyswMFBeXl4qU6aMWrRoYUx2ytG8eXMtXbpUnTp1UoUKFeTt7a0qVaooKipKS5cuzXOzzRsVFham5cuXKyoqSpUrV5a3t7cqVqyoDh066PPPP1fLli0L/Nx169bVZ599pjvvvFP+/v4qW7asHnvsMX300UdGm9zLXzn6GXUtDz/8sD777DPdf//9CgoKkre3t0qWLKkmTZpo7Nixeu211wrcJyA3k8WZdwAAAOAWcOLECWNm2qOPPmrcGR5FKzY2Vunp6apSpYqCgoKsitcdOnTQsWPH1KRJEy1ZssSFUQIAgJvdnXfeqbNnzyo0NFSbN292dTi3pJUrV2rEiBGSZCxLhKL39ddfq1y5cqpSpYoqV65szLo/c+aMWrduLSl7otPVV+gC7oKlXQAAgFvauHGjFi5cKElq27atcRnqtm3bdOzYMUlyeI1FAAAAe33zzTf67rvvdPbsWUnZVxYAt7Jp06bpzz//lJS9fM2jjz6qv//+27g/lcT4HO6NQjoAAHBL3bt31/Lly3X58mVt3brVuDFUjlKlSuVZ7xIAAMBZJk+ebHx5L+mGlsQAbgZ9+vTRqFGjJEkxMTGKiYmx2l6vXj21a9fOFaEBTkEhHQAAuKU6depo0aJFmj17tvbs2WPMBqtQoYKaN2+uAQMGFPhmvQAAALakp6dLyl7vuXTp0urUqdMN3fsFuBl069ZNZcqU0aJFi3TgwAFdvHhRXl5eCg4OVtu2bfX888/Ly4tSJNwXa6QDAAAAyFdiYqJef/11/fzzzypXrpz69Olj3Kh6//79GjNmjP7880/VqlVLb7zxhho2bGjsu379ek2bNk3Jycm66667NG7cuHxv2gwAAAC4Aw9XBwAAAACgeHrxxRfl7++vlStXauTIkZo2bZq++uorXbp0Sf369VNkZKRWrlypiIgI9e/fX5cuXZIk7du3T9HR0Ro8eLCWLFmiixcvGjeBAwAAANwRM9IBAAAA5HHhwgU1b95c69atU+3atSVJQ4YMUcWKFVW/fn3NnDlTW7ZskclkksViUYcOHTRgwABFRUVp2LBh8vDw0KRJkyRlz2xv27atvvrqK5ZcAgAAgFtiRjoAAACAPEqUKCE/Pz+tXLlSGRkZOnLkiHbv3q169eopLi5OzZo1k8lkkiSZTCY1bdpUe/fulSTFxcUpMjLSeK4qVaooKChIcXFxrugKAAAAcMNY4d+G5OS/XHZsHx9PpaebXXZ8OI6cuR9y5n7ImfshZ+6HnLlGxYqlXR1CHr6+vho9erTGjRunBQsWyGw2KyoqSt26ddPXX3+tWrVqWbUvX768Dh48KEk6c+aMKlWqlGd7UlKS3cdPTv5L/1enL1Le3p7KyODfgLsif+6L3Lk38ue+yJ37InfOU6GCfWNxCunFUM4Jg8kksfCOeyBn7oecuR9y5n7ImfshZ7ja4cOH1bZtWz3zzDM6ePCgxo0bp5YtWyotLU0+Pj5WbX18fJSeni5Junz5ss3t9vDx8bzxDjjIZJI8PT35N+CmyJ/7Infujfy5L3Lnvsida1BIBwAAAJDHrl27tHz5cm3fvl0lSpRQo0aNdPr0ac2cOVMhISF5iuLp6ekqUaKEpOzZ7Plt9/Pzs/v46enmIp+RnnMymplp5qTUDZE/90Xu3Bv5c1/kzn2RO9coNmukp6enq3Pnzvrxxx+NxxISEvT000+rSZMmeuCBB7Rz506rfb7//nt17txZ4eHh6tOnjxISEqy2z5s3T61bt1ZERIRGjhyptLS0IukLAAAA4O5+/fVXVa9e3SiOS1L9+vV16tQpBQYGKiUlxap9SkqKsZzLtbZXrFjRoRgslqL/cdVx+SF/t/oPuXPvH/Lnvj/kzn1/yJ1zX0t7FItC+pUrVzR06FBjTUVJslgsGjRokCpUqKAVK1bokUce0eDBg3Xq1ClJ0qlTpzRo0CBFRUVp+fLlCggI0MCBA2X5v95v3rxZM2bM0NixYzV//nzFxcVp6tSpLukfAAAA4G4qVaqk48ePW80sP3LkiKpWrarw8HDt2bPHGHtbLBbt3r1b4eHhkqTw8HDFxsYa+yUmJioxMdHYDgAAALgblxfSDx06pO7duys+Pt7q8R9++EEJCQkaO3aswsLC1L9/fzVp0kQrVqyQJC1btkwNGzZU3759dfvtt2vixIk6efKkfvrpJ0nSggUL9NRTT6lt27Zq3Lix3njjDa1YsYJZ6QAAAIAd2rVrJ29vb40aNUpHjx7VN998ow8//FC9e/dWx44ddfHiRU2YMEGHDh3ShAkTlJaWpk6dOkmSevXqpTVr1mjZsmU6cOCAhg0bpjZt2igkJMTFvQIAAAAKxuWF9J9++kktWrTQkiVLrB6Pi4tT/fr15e/vbzzWrFkz7d2719geGRlpbPPz81ODBg20d+9emc1m/fLLL1bbmzRpooyMDB04cKBwOwQAAADcBEqXLq158+YpOTlZjz32mCZOnKjnn39ePXr0UKlSpTRr1izFxsYqKipKcXFxmj17tjF2j4iI0NixYxUTE6NevXqpbNmymjhxoot7BAAAABScy282+vjjj+f7eHJysrHGYo7y5csrKSnputsvXryoK1euWG338vJSuXLljP0BAAAA2FarVi198skn+W5r3LixVq1adc19o6KiFBUVVVihAQAAAEXK5YX0a0lLS5OPj4/VYz4+PsYajba2X7582fj9Wvvby2RyNPIbl3NMVxwbBUPO3A85cz/kzP2QM/dDzgAAAAAgf8W2kO7r66vz589bPZaenq4SJUoY268uiqenp6tMmTLy9fU1fr96u5+fn90x+Ph4FiDyG2cySZ6enjKZHLtzLFyHnLkfcuZ+yJn7IWfuh5wBAAAAQP6KbSE9MDBQhw4dsnosJSXFWK4lMDBQKSkpebbXq1dP5cqVk6+vr1JSUhQWFiZJyszM1Pnz51WxYkW7Y0hPN7tsRrrFImVmmjmJdRPkzP2QM/dDztwPOXM/5AwAAAAA8ldsC+nh4eGaPXu2Ll++bMxCj42NVbNmzYztsbGxRvu0tDTt379fgwcPloeHhxo1aqTY2Fi1aNFCkrR37155eXmpbt26DsXhypNIi4XZYO6GnLkfcuZ+yJn7IWfuh5wBAAAAgDUPVwdwLc2bN1eVKlU0YsQIHTx4ULNnz9a+ffv02GOPSZK6du2q3bt3a/bs2Tp48KBGjBihqlWrGoXzxx9/XHPnztWWLVu0b98+vf766+revbtDS7sAAAAAAAAAAFBsC+menp764IMPlJycrKioKK1du1YxMTEKCgqSJFWtWlXvv/++VqxYoccee0znz59XTEyMTP+3FsuDDz6o/v37a/To0erbt68aN26sV155xZVdAgAAAAAAAAC4IZPFwoW715Kc/JdLjmsySd7ensrIYH1Sd0HO3A85cz/kzP2QM/dDzlynYsXSrg6h2HHFWJx/A+6N/LkvcufeyJ/7Infui9w5l71j8WI7Ix0AAAAAAAAAgOKAQjoAAAAAAAAAADZQSAcAAAAAAAAAwAYK6QAAAAAAAAAA2EAhHQAAAAAAAAAAGyikAwAAAAAAAABgA4V0AAAAAAAAAABsoJAOAAAAAAAAAIANXq4OAHllZmYqPv6oMjLMdu9TrVqovLxIJwAAAFBQBRmHS4zFAQAAbgWM9oqh48ePae3vxxQQWNWu9imJJ9RFUs2atQo1LgAAAOBmdvz4MZ1euVRBAQGSxb59TqSelbr2YCwOAABwk6OQXkyVr1JVgdXCXB0GAAAAcEupWr68agQG2l1IlyTH5q8DAADAHbFGOgAAAAAAAAAANlBIBwAAAAAAAADABgrpAAAAAAAAAADYQCEdAAAAAAAAAAAbKKQDAAAAAAAAAGADhXQAAAAAAAAAAGygkA4AAAAAAAAAgA0U0gEAAAAAAAAAsIFCOgAAAAAAAAAANlBIBwAAAAAAAADABgrpAAAAAAAAAADYQCEdAAAAAAAAAAAbKKQDAAAAAAAAAGADhXQAAAAAAAAAAGygkA4AAAAAAAAAgA0U0gEAAAAAAAAAsIFCOgAAAAAAAAAANlBIBwAAAAAAAADABgrpAAAAAAAAAADYQCEdAAAAAAAAAAAbKKQDAAAAAAAAAGADhXQAAAAAAAAAAGygkA4AAAAAAAAAgA0U0gEAAAAAAAAAsIFCOgAAAAAAAAAANlBIBwAAAAAAAADABgrpAAAAAAAAAADYQCEdAAAAAAAAAAAbKKQDAAAAAAAAAGADhXQAAAAAAAAAAGygkA4AAAAAAAAAgA0U0gEAAAAAAAAAsIFCOgAAAIA8Vq5cqTp16uT5qVu3riRp//796tatm8LDw9W1a1f9+uuvVvuvX79e7du3V3h4uAYNGqTU1FRXdAMAAABwCgrpAAAAAPJ44IEHtHPnTuNn27Ztql69uvr06aNLly6pX79+ioyM1MqVKxUREaH+/fvr0qVLkqR9+/YpOjpagwcP1pIlS3Tx4kWNGDHCxT0CAAAACo5COgAAAIA8SpQooYoVKxo/a9eulcVi0csvv6yNGzfK19dXw4YNU1hYmKKjo1WyZElt2rRJkrRw4UJ16tRJXbp0Ud26dTVlyhRt375dCQkJLu4VAAAAUDAU0gEAAADYdP78ec2ZM0cvvfSSfHx8FBcXp2bNmslkMkmSTCaTmjZtqr1790qS4uLiFBkZaexfpUoVBQUFKS4uzhXhAwAAADeMQjoAAAAAmxYtWqRKlSqpY8eOkqTk5GRVqlTJqk358uWVlJQkSTpz5ozN7QAAAIC78XJ1AAAAAACKL4vFomXLlum5554zHktLS5OPj49VOx8fH6Wnp0uSLl++bHO7vf5vwnuRsTqevcc25bMvXMJELtwWuXNv5M99kTv3Re5cg0I6AAAAgGv65ZdfdPr0aT344IPGY76+vnmK4unp6SpRooTN7X5+fnYf18fH8waiLhhvb0/JJHk4cFbqYTLJw9sze1+4lMkkeXp6ymSSLBZXRwNHkDv3Rv7cF7lzX+TONSikAwAAALimb7/9VpGRkSpbtqzxWGBgoFJSUqzapaSkGMu5XGt7xYoV7T5uerq5yGdZZWSY5WWRshw4I82yWGTOMCsjw1yIkcEeOcWEzEwzRQU3Q+7cG/lzX+TOfZE716CQDgAAAOCa9u3bp6ZNm1o9Fh4erjlz5shischkMslisWj37t0aMGCAsT02NlZRUVGSpMTERCUmJio8PNyhYxf1iaHV8ew9tiWffeFSFgv5cFfkzr2RP/dF7twXuSta3GwUAAAAwDUdPHhQtWrVsnqsY8eOunjxoiZMmKBDhw5pwoQJSktLU6dOnSRJvXr10po1a7Rs2TIdOHBAw4YNU5s2bRQSEuKKLgAAAAA3jEI6AAAAgGtKSUlRmTJlrB4rVaqUZs2aZcw6j4uL0+zZs+Xv7y9JioiI0NixYxUTE6NevXqpbNmymjhxoivCBwAAAJyCpV0AAAAAXNO+ffvyfbxx48ZatWrVNfeLiooylnYBAAAA3B0z0gEAAAAAAAAAsIFCOgAAAAAAAAAANlBIBwAAAAAAAADABgrpAAAAAAAAAADYQCEdAAAAAAAAAAAbKKQDAAAAAAAAAGADhXQAAAAAAAAAAGygkA4AAAAAAAAAgA0U0gEAAAAAAAAAsIFCOgAAAAAAAAAANlBIBwAAAAAAAADABgrpAAAAAAAAAADYQCEdAAAAAAAAAAAbKKQDAAAAAAAAAGADhXQAAAAAAAAAAGygkA4AAAAAAAAAgA0U0gEAAAAAAAAAsIFCOgAAAAAAAAAANlBIBwAAAAAAAADABgrpAAAAAAAAAADYQCEdAAAAAAAAAAAbKKQDAAAAAAAAAGADhXQAAAAAAAAAAGygkA4AAAAAAAAAgA0U0gEAAAAAAAAAsIFCOgAAAAAAAAAANlBIBwAAAAAAAADABgrpAAAAAAAAAADYQCEdAAAAAAAAAAAbKKQDAAAAAAAAAGADhXQAAAAAAAAAAGygkA4AAAAAAAAAgA3FvpCemJio/v37q2nTpmrXrp3mzZtnbNu/f7+6deum8PBwde3aVb/++qvVvuvXr1f79u0VHh6uQYMGKTU1tYijBwAAAAAAAAC4u2JfSH/xxRfl7++vlStXauTIkZo2bZq++uorXbp0Sf369VNkZKRWrlypiIgI9e/fX5cuXZIk7du3T9HR0Ro8eLCWLFmiixcvasSIES7uDQAAAAAAAADA3RTrQvqFCxe0d+9ePf/88woNDVX79u3VunVr7dq1Sxs3bpSvr6+GDRumsLAwRUdHq2TJktq0aZMkaeHCherUqZO6dOmiunXrasqUKdq+fbsSEhJc3CsAAAAAAAAAgDsp1oX0EiVKyM/PTytXrlRGRoaOHDmi3bt3q169eoqLi1OzZs1kMpkkSSaTSU2bNtXevXslSXFxcYqMjDSeq0qVKgoKClJcXJwrugIAAAAAAAAAcFPFupDu6+ur0aNHa8mSJQoPD1enTp109913q1u3bkpOTlalSpWs2pcvX15JSUmSpDNnztjcDgAAAAAAAACAPbxcHcD1HD58WG3bttUzzzyjgwcPaty4cWrZsqXS0tLk4+Nj1dbHx0fp6emSpMuXL9vcbq//m/BepHKOaZJkKcB+KHpGzsiB2yBn7oecuR9y5n7IGQAAAADkr1gX0nft2qXly5dr+/btKlGihBo1aqTTp09r5syZCgkJyVMUT09PV4kSJSRlz2bPb7ufn5/dx/fx8bzxThSAt7enTCazTB7ZxfTrMXlk7+Pt7Zp4kV1w8PT0lMkkWRz59gMuQ87cDzlzP+TM/ZAzAAAAAMhfsS6k//rrr6pevbpRHJek+vXr68MPP1RkZKRSUlKs2qekpBjLuQQGBua7vWLFinYfPz3d7JIZWRkZZlksFlmy7JuRbsnK3icjw1zosSF/OQWHzEwzhQc3Qc7cDzlzP+TM/ZAzAAAAAMhfsS6kV6pUScePH1d6erqxTMuRI0dUtWpVhYeHa86cObJYLDKZTLJYLNq9e7cGDBggSQoPD1dsbKyioqIkSYmJiUpMTFR4eLhDMbjiJDLnmI4emhNe17NYyIO7IWfuh5y5H3LmfsgZAAAAAFgr1jcbbdeunby9vTVq1CgdPXpU33zzjT788EP17t1bHTt21MWLFzVhwgQdOnRIEyZMUFpamjp16iRJ6tWrl9asWaNly5bpwIEDGjZsmNq0aaOQkBAX9woAAAAAAAAA4E6KdSG9dOnSmjdvnpKTk/XYY49p4sSJev7559WjRw+VKlVKs2bNMmadx8XFafbs2fL395ckRUREaOzYsYqJiVGvXr1UtmxZTZw40cU9AgAAAAAAAAC4m2K9tIsk1apVS5988km+2xo3bqxVq1Zdc9+oqChjaRcAAAAAAAAAAAqiWM9IBwAAAAAAAADA1SikAwAAAAAAAABgA4V0AAAAAAAAAABsoJAOAAAAAAAAAIANFNIBAAAAAAAAALCBQjoAAAAAAAAAADZQSAcAAAAAAAAAwAYK6QAAAADylZ6erjfeeEP/+te/dOedd+qdd96RxWKRJO3fv1/dunVTeHi4unbtql9//dVq3/Xr16t9+/YKDw/XoEGDlJqa6oouAAAAAE5BIR0AAABAvsaPH6/vv/9ec+fO1dtvv62lS5dqyZIlunTpkvr166fIyEitXLlSERER6t+/vy5duiRJ2rdvn6KjozV48GAtWbJEFy9e1IgRI1zcGwAAAKDgvFwdAAAAAIDi5/z581qxYoU++eQTNW7cWJLUt29fxcXFycvLS76+vho2bJhMJpOio6O1Y8cObdq0SVFRUVq4cKE6deqkLl26SJKmTJmitm3bKiEhQSEhIS7sFQAAAFAwzEgHAAAAkEdsbKxKlSql5s2bG4/169dPEydOVFxcnJo1ayaTySRJMplMatq0qfbu3StJiouLU2RkpLFflSpVFBQUpLi4uCLtAwAAAOAsFNIBAAAA5JGQkKDg4GCtXr1aHTt21L333quYmBhlZWUpOTlZlSpVsmpfvnx5JSUlSZLOnDljczsAAADgbljaBQAAAEAely5d0vHjx7V48WJNnDhRycnJGj16tPz8/JSWliYfHx+r9j4+PkpPT5ckXb582eZ2e/3fhPciY3U8e49tymdfuISJXLgtcufeyJ/7Infui9y5BoV0AAAAAHl4eXnp77//1ttvv63g4GBJ0qlTp7Ro0SJVr149T1E8PT1dJUqUkCT5+vrmu93Pz8/u4/v4eN5gDxzn7e0pmSQPB85KPUwmeXh7Zu8LlzKZJE9PT5lMksXi6mjgCHLn3sif+yJ37ovcuQaFdAAAAAB5VKxYUb6+vkYRXZJq1KihxMRENW/eXCkpKVbtU1JSjOVcAgMD891esWJFu4+fnm4u8llWGRlmeVmkLAfOSLMsFpkzzMrIMBdiZLBHTjEhM9NMUcHNkDv3Rv7cF7lzX+TONSikAwAAAMgjPDxcV65c0dGjR1WjRg1J0pEjRxQcHKzw8HDNmTNHFotFJpNJFotFu3fv1oABA4x9Y2NjFRUVJUlKTExUYmKiwsPDHYqhqE8MrY5n77Et+ewLl7JYyIe7Infujfy5L3Lnvshd0eJmowAAAADyqFmzptq0aaMRI0bowIED+vbbbzV79mz16tVLHTt21MWLFzVhwgQdOnRIEyZMUFpamjp16iRJ6tWrl9asWaNly5bpwIEDGjZsmNq0aaOQkBAX9woAAAAoGArpAAAAAPL11ltvqVq1aurVq5eGDx+uJ554Qr1791apUqU0a9YsY9Z5XFycZs+eLX9/f0lSRESExo4dq5iYGPXq1Utly5bVxIkTXdwbAAAAoOBY2gUAAABAvkqXLq0pU6bku61x48ZatWrVNfeNiooylnYBAAAA3B0z0gEAAAAAAAAAsIFCOgAAAAAAAAAANlBIBwAAAAAAAADABgrpAAAAAAAAAADYQCEdAAAAAAAAAAAbKKQDAAAAAAAAAGADhXQAAAAAAAAAAGygkA4AAAAAAAAAgA0U0gEAAAAAAAAAsIFCOgAAAAAAAAAANlBIBwAAAAAAAADABgrpAAAAAAAAAADYQCEdAAAAAAAAAAAbKKQDAAAAAAAAAGADhXQAAAAAAAAAAGygkA4AAAAAAAAAgA0U0gEAAAAAAAAAsIFCOgAAAAAAAAAANlBIBwAAAAAAAADABgrpAAAAAAAAAADYQCEdAAAAAAAAAAAbKKQDAAAAAAAAAGADhXQAAAAAAAAAAGygkA4AAAAAAAAAgA0U0gEAAAAAAAAAsIFCOgAAAAAAAAAANlBIBwAAAAAAAADABgrpAAAAAAAAAADYQCEdAAAAAAAAAAAbKKQDAAAAAAAAAGADhXQAAAAAAAAAAGygkA4AAAAAAAAAgA0U0gEAAAAAAAAAsIFCOgAAAAAAAAAANlBIBwAAAAAAAADABgrpAAAAAAAAAADYQCEdAAAAAAAAAAAbKKQDAAAAAAAAAGADhXQAAAAAAAAAAGygkA4AAAAAAAAAgA0U0gEAAAAAAAAAsIFCOgAAAAAAAAAANlBIBwAAAAAAAADABgrpAAAAAAAAAADYQCEdAAAAAAAAAAAbKKQDAAAAAAAAAGADhXQAAAAAAAAAAGygkA4AAAAAAAAAgA0U0gEAAAAAAAAAsIFCOgAAAAAAAAAANlBIBwAAAAAAAADABgrpAAAAAPL11VdfqU6dOlY/L7zwgiRp//796tatm8LDw9W1a1f9+uuvVvuuX79e7du3V3h4uAYNGqTU1FRXdAEAAABwCgrpAAAAAPJ16NAhtW3bVjt37jR+xo8fr0uXLqlfv36KjIzUypUrFRERof79++vSpUuSpH379ik6OlqDBw/WkiVLdPHiRY0YMcLFvQEAAAAKjkI6AAAAgHwdPnxYtWvXVsWKFY2fMmXKaOPGjfL19dWwYcMUFham6OholSxZUps2bZIkLVy4UJ06dVKXLl1Ut25dTZkyRdu3b1dCQoKLewQAAAAUDIV0AAAAAPk6fPiwQkND8zweFxenZs2ayWQySZJMJpOaNm2qvXv3GtsjIyON9lWqVFFQUJDi4uKKImwAAADA6SikAwAAAMjDYrHo6NGj2rlzpzp06KD27dvrrbfeUnp6upKTk1WpUiWr9uXLl1dSUpIk6cyZMza3AwAAAO7Gy9UBAAAAACh+Tp06pbS0NPn4+GjatGk6ceKExo8fr8uXLxuP5+bj46P09HRJ0uXLl21ut9f/TXgvMlbHs/fYpnz2hUuYyIXbInfujfy5L3Lnvsida1BIBwAAAJBHcHCwfvzxR5UtW1Ymk0n16tVTVlaWXnnlFTVv3jxPUTw9PV0lSpSQJPn6+ua73c/Pz+7j+/h43ngnHOTt7SmZJA8Hzko9TCZ5eHtm7wuXMpkkT09PmUySxeLqaOAIcufeyJ/7Infui9y5BoV0AAAAAPkqV66c1e9hYWG6cuWKKlasqJSUFKttKSkpxnIugYGB+W6vWLGi3cdOTzcX+SyrjAyzvCxSlgNnpFkWi8wZZmVkmAslpszMTB0/fsyhfapXD5WX1613qpdTTMjMNFNUcDPkzr2RP/dF7twXuXONW290BQAAAOC6vv32W7388svatm2bMZP8999/V7ly5dSsWTPNmTNHFotFJpNJFotFu3fv1oABAyRJ4eHhio2NVVRUlCQpMTFRiYmJCg8PdyiGoj4xtDqevce25LOvEx0/fkyJK5aoakB5u9qfSD0rde2hmjVrFU5AbsBiYXaeuyJ37o38uS9y577IXdGikA4AAAAgj4iICPn6+mrUqFEaNGiQEhISNGXKFD333HPq2LGj3n77bU2YMEE9e/bU4sWLlZaWpk6dOkmSevXqpd69e6tJkyZq1KiRJkyYoDZt2igkJMTFvXJPVQPKK6xyZbvbF87ceAAAgFubh6sDAAAAAFD8lCpVSnPnzlVqaqq6du2q6Oho9ejRQ88995xKlSqlWbNmGbPO4+LiNHv2bPn7+0vKLsKPHTtWMTEx6tWrl8qWLauJEye6uEcAAABAwTEjHQAAAEC+br/9dn3yySf5bmvcuLFWrVp1zX2joqKMpV0AAAAAd8eMdAAAAAAAAAAAbKCQDgAAAAAAAACADRTSAQAAAAAAAACwgUI6AAAAAAAAAAA2UEgHAAAAAAAAAMCGYl9IT09P1xtvvKF//etfuvPOO/XOO+/IYrFIkvbv369u3bopPDxcXbt21a+//mq17/r169W+fXuFh4dr0KBBSk1NdUUXAAAAAAAAAABurNgX0sePH6/vv/9ec+fO1dtvv62lS5dqyZIlunTpkvr166fIyEitXLlSERER6t+/vy5duiRJ2rdvn6KjozV48GAtWbJEFy9e1IgRI1zcGwAAAAAAAACAu/FydQC2nD9/XitWrNAnn3yixo0bS5L69u2ruLg4eXl5ydfXV8OGDZPJZFJ0dLR27NihTZs2KSoqSgsXLlSnTp3UpUsXSdKUKVPUtm1bJSQkKCQkxIW9AgAAAAAAAAC4k2I9Iz02NlalSpVS8+bNjcf69euniRMnKi4uTs2aNZPJZJIkmUwmNW3aVHv37pUkxcXFKTIy0tivSpUqCgoKUlxcXJH2AQAAAAAAAADg3op1IT0hIUHBwcFavXq1OnbsqHvvvVcxMTHKyspScnKyKlWqZNW+fPnySkpKkiSdOXPG5nYAAAAAAAAAAOxRrJd2uXTpko4fP67Fixdr4sSJSk5O1ujRo+Xn56e0tDT5+PhYtffx8VF6erok6fLlyza32+v/JrwXqZxjmiRZCrAfip6RM3LgNsiZ+yFn7oecuR9yBgAAAAD5K9aFdC8vL/399996++23FRwcLEk6deqUFi1apOrVq+cpiqenp6tEiRKSJF9f33y3+/n52X18Hx/PG+xBwXh7e8pkMsvkkV1Mvx6TR/Y+3t6uiRfZBQdPT0+ZTJLFkW8/4DLkzP2QM/dDztwPOQMAAACA/BXrQnrFihXl6+trFNElqUaNGkpMTFTz5s2VkpJi1T4lJcVYziUwMDDf7RUrVrT7+OnpZpfMyMrIMMtisciSZd+MdEtW9j4ZGeZCjw35yyk4ZGaaKTy4CXLmfsiZ+yFn7oecAQAAAED+inUhPTw8XFeuXNHRo0dVo0YNSdKRI0cUHBys8PBwzZkzRxaLRSaTSRaLRbt379aAAQOMfWNjYxUVFSVJSkxMVGJiosLDwx2KwRUnkTnHdPTQnPC6nsVCHtwNOXM/5Mz9kDP3Q84AAAAAwFqxvtlozZo11aZNG40YMUIHDhzQt99+q9mzZ6tXr17q2LGjLl68qAkTJujQoUOaMGGC0tLS1KlTJ0lSr169tGbNGi1btkwHDhzQsGHD1KZNG4WEhLi4VwAAAAAAAAAAd1KsZ6RL0ltvvaVx48apV69e8vPz0xNPPKHevXvLZDJp1qxZGjNmjJYuXao6depo9uzZ8vf3lyRFRERo7Nixmj59ui5cuKBWrVpp3LhxLu4NAAAAgJtJptmshPh4h/apVi1UXl7F/lQMAAAAuRT70Vvp0qU1ZcqUfLc1btxYq1atuua+UVFRxtIuAAAAAOBsp86lKuvbrfIMCr5+Y0knUs9KXXuoZs1ahRwZAAAAnKnYF9IBAAAAoDgLCghQWOXKdrc3F2IsAAAAKBzFeo10AAAAAAAAAABcrUCF9PXr1ys9Pd3ZsQAAAAC4QYzVAQAAAOcrUCF92LBhatWqlV5//XXt27fP2TEBAAAAKCDG6gAAAIDzFaiQ/s0336hv37764Ycf1KNHDz3wwAOaO3eukpOTnR0fAAAAAAcwVgcAAACcr0CF9MqVK+v555/Xpk2b9NlnnykyMlJz5sxR27ZtNWDAAH355ZfKzMx0dqwAAAAAroOxOgAAAOB8Xjf6BE2bNlXTpk3VrVs3TZkyRdu2bdO2bdtUoUIFPfXUU+rbt688PT2dESsAAAAABzBWBwAAAJzjhgrpJ0+e1Jo1a7RmzRrFx8erWrVqGjp0qNq0aaNt27YpJiZGhw4d0uTJk50VLwAAAAA7MFYHAAAAnKdAhfRly5ZpzZo12r17t3x9fdWxY0dNmDBBkZGRRpvatWvr3LlzWrx4MYNzAAAAoIgwVgcAAACcr0CF9Ndee03h4eF6/fXX9cADD6hUqVL5tqtTp4569OhxQwECAAAAsB9jdQAAAMD5ClRIX79+vWrVqiWz2WysqXj58mVlZGSodOnSRrsuXbo4JUgAAAAA9mGsDgAAADifR0F2Cg0N1ZgxY9S9e3fjsd27d6tly5aaPHmysrKynBYgAAAAAPsxVgcAAACcr0CF9OnTp2vt2rXq3Lmz8Vj9+vX18ssva+nSpfroo4+cFiAAAAAA+zFWBwAAAJyvQEu7rFu3TsOHD1fPnj2Nx8qVK6enn35aXl5eWrBggfr16+e0IAEAAADYh7E6AAAA4HwFKqSfO3dOISEh+W6rWbOmkpKSbigoOMZszlR8/CmH9qlWLVReXgVKPwAAAIoxxuoAAACA8xWoklqzZk1t3rxZrVq1yrPtm2++UfXq1W84MNgvNemUkjLSFV8q0672KYkn1EVSzZq1CjUuAAAAFD3G6gAAAIDzFaiQ3qdPH7366qs6f/682rdvr/Llyys1NVVbt27VF198oYkTJzo7TlxHQOVgVQ4Nc3UYAAAAcDHG6gAAAIDzFaiQ3qVLF/3zzz/64IMP9OWXXxqP33bbbXrttdfUpUsXZ8UHAAAAwAGM1QEAAADnK/Ai2U888YQef/xxHT16VOfPn1eZMmVUs2ZNeXh4ODM+AAAAAA5irA4AAAA41w3dbdJkMqlmzZrOigUAAACAkzBWBwAAAJynQIX01NRUTZgwQdu2bVNaWposFovVdpPJpP379zslQAAAAAD2Y6xevGWazUqIj7e7fXx8vEKVVYgRAQAAwB4FKqSPHTtWW7du1YMPPqjKlStziSgAAABQTDBWL95OnUtV1rdb5RkUbFf75MMHVTm4aiFHBQAAgOspUCF9x44dGjlypHr06OHseAAAAADcAMbqxV9QQIDCKle2q218SnIhRwMAAAB7FGh6ire3t0JCQpwdCwAAAIAbxFgdAAAAcL4CFdLvu+8+rV+/3tmxAAAAALhBjNUBAAAA5yvQ0i7169fXtGnTlJCQoPDwcJUoUcJqu8lk0qBBg5wSIAAAAAD7MVYHAAAAnK/ANxuVpJ9//lk///xznu0MzgEAAADXYKwOAAAAOF+BCukHDhxwdhwAAAAAnICxOgAAAOB8BVojPbe//vpLhw8fVnp6usxmszNiAgAAAOAEjNUBAAAA5yhwIf3HH39Ut27d1Lx5cz300EM6ePCgXnrpJU2aNMmZ8QEAAABwEGN1AAAAwLkKVEjftWuXnn32WZUoUUIvv/yyLBaLJKlu3bpasGCBPvnkE6cGCQAAAMA+jNVvbZlms+Lj43XkyCG7fzIzM10dNgAAQLFXoDXSp02bpnvvvVfvvfeeMjMzNXXqVEnSgAEDdOnSJS1btkzPPPOMUwMFAAAAcH2M1W9tp86lKuvbrfIMCrar/YnUs1LXHqpZs1YhRwYAAODeClRI//333zVo0CBJkslkstrWqlUrzZ8//8YjAwAAAOAwxuoICghQWOXKdrdn9XwAAIDrK9DSLqVLl1ZycnK+2xITE1W6dOkbCgoAAABAwTBWBwAAAJyvQDPS7733Xr377ruqXbu26tevLyl7tktSUpI+/PBDtWnTxpkxAgAAALBTYYzV+/Xrp4CAAONmpfv379eYMWP0559/qlatWnrjjTfUsGFDo/369es1bdo0JScn66677tK4ceMUEBDglP7BuTLNZiXExzu0T7VqofLyKtCpJAAAgNsq0OjnpZdeUlxcnLp3764KFSpIkoYOHaqkpCRVqVJFQ4cOdWqQAAAAAOzj7LH6hg0btH37dj366KOSpEuXLqlfv3566KGHNGnSJC1atEj9+/fXV199JX9/f+3bt0/R0dF64403VLduXU2YMEEjRozQrFmznN5X3DjWVAcAALBPgQrpZcuW1bJly7R69Wr98MMPOn/+vEqXLq3evXsrKipKfn5+zo4TAAAAgB2cOVY/f/68pkyZokaNGhmPbdy4Ub6+vho2bJhMJpOio6O1Y8cObdq0SVFRUVq4cKE6deqkLl26SJKmTJmitm3bKiEhQSEhIc7uLpyANdUBAACur8DX4/n4+Kh79+7q3r27M+MBAAAAcIOcNVafPHmyHnnkEZ05c8Z4LC4uTs2aNTNuZGoymdS0aVPt3btXUVFRiouL07///W+jfZUqVRQUFKS4uDgK6QAAAHBbBSqkr169+rptcmagAAAAACg6zhqr79q1S//73/+0bt06vf7668bjycnJqlXLelmP8uXL6+DBg5KkM2fOqFKlSnm2JyUlXfeYV/u/Wn2RsTqevcc2XfXnLdK+qHNjD1Mxjg22kTv3Rv7cF7lzX+TONQpUSH/11VfzfdxkMsnT01Oenp4U0gEAAAAXcMZY/cqVKxozZoxGjx6tEiVKWG1LS0uTj4+P1WM+Pj5KT0+XJF2+fNnmdnv5+Hg61N4ZvL09JZPk4cBZqYfJJJOH/fvcDO09vD2zX6tixmSSPD09ZTJJFouro4EjyJ17I3/ui9y5L3LnGgUqpH/99dd5Hrt06ZL+97//ac6cOYqJibnhwAAAAAA4zhlj9RkzZqhhw4Zq3bp1nm2+vr55iuLp6elGwf1a2x29j1J6urnIZ1llZJjlZZGyHDgjzbJYZMmyf5+bob05w6yMjOK3UnpOMSEz00xRwc2QO/dG/twXuXNf5M41ClRIDw7O/47ut99+uzIyMjRu3Dh9/vnnNxQYAAAAAMc5Y6y+YcMGpaSkKCIiQpKMwvjmzZvVuXNnpaSkWLVPSUkxlnMJDAzMd3vFihUd7ktRnxhaHc/eY1uu+vMWaV+cT9otluIdH66N3Lk38ue+yJ37IndFy8PZT1inTh399ttvzn5aAAAAADfI3rH6p59+qnXr1mn16tVavXq12rVrp3bt2mn16tUKDw/Xnj17ZPm/szaLxaLdu3crPDxckhQeHq7Y2FjjuRITE5WYmGhsBwAAANxRgWakX0t6erqWL1+u8uXLO/NpAQAAANwgR8bqV89qL1mypCSpevXqKl++vN5++21NmDBBPXv21OLFi5WWlqZOnTpJknr16qXevXurSZMmatSokSZMmKA2bdooJCTE+Z0CAAAAikiBCunt2rWT6aoFC7OysnTu3DlduXJFw4cPd0pwAAAAABxT2GP1UqVKadasWRozZoyWLl2qOnXqaPbs2fL395ckRUREaOzYsZo+fbouXLigVq1aady4cTd0TAAAAMDVClRIb968eZ7BuZQ9qG7btq3uvPPOGw4MAAAAgOMKY6w+adIkq98bN26sVatWXbN9VFSUoqKiHD4OAAAAUFwVqJB+9UAaAAAAQPHAWB0AAABwvgIV0k+dOuVQ+6CgoIIcBgAAAICDGKsDAAAAzue0NdJt+f333wtyGAAAAAAOYqwOAAAAOF+BCunTpk3TmDFj1KBBAz388MMKDAzUuXPn9M033+iLL77Q888/r+DgYGfHCgAAAOA6GKsDAAAAzlegQvqaNWvUtm3bPOsvPvDAAypfvrx2796twYMHOyVAAAAAAPZjrA4AAAA4n0dBdtq1a5c6d+6c77a7775bsbGxNxQUAAAAgIJhrA4AAAA4X4EK6bfddpvi4uLy3bZr1y4FBgbeUFAAAAAACoaxOgAAAOB8BVra5bHHHtPMmTOVlpamdu3aKSAgQCkpKdq0aZMWLVqk1157zdlxAgAAALADY3UAAADA+QpUSB84cKD++usvzZs3T3PnzpUkWSwW+fn56b///a969uzp1CABAAAA2IexOgAAAOB8BSqkm0wmvfrqqxo4cKD27t2rCxcu6LbbblOTJk1UqlQpZ8cIAAAAwE6M1QEAAADnK1AhPUepUqVUqVIlSVKTJk2UmZnplKAAAAAA3BjG6gAAAIDzFLiQvmbNGr399ttKTk6WyWTSsmXL9P7778vb21tvv/22fHx8nBknAAAAADsxVgcAAACcy6MgO23cuFHDhw/XHXfcoXfeeUdZWVmSpPvuu0/bt2/XBx984NQgAQAAANiHsToKU6bZrPj4eB05csihH66IAAAA7q5AM9I//PBD9ezZU6+//rrMZrPxeNeuXZWamqqlS5fqxRdfdFaMAAAAAOzEWB2F6dS5VGV9u1WeQcF273Mi9azUtYdq1qxViJEBAAAUrgIV0o8eParhw4fnuy08PFzvv//+DQUFAAAAoGAYq6OwBQUEKKxyZYf2MV+/CQAAQLFWoKVdypcvr8OHD+e77fDhwypfvvwNBQUAAACgYBirAwAAAM5XoBnpDzzwgKZPn65KlSrpnnvukSSZTCb9+uuv+uCDD9S5c2enBgnnMpszFR9/yqF9qlULlZdXge9NCwAAgCLCWB0AAABwvgJVRl988UX9+eefevHFF+XhkT2pvXfv3rp06ZIiIyP1n//8x6lBwrlSk04pKSNd8aXsu+FPSuIJdZFY0xAAAMANMFYHAAAAnK9AhXQfHx999NFH+u677/TDDz/o/PnzKl26tJo3b6577rlHJpPJ2XHCyQIqB6tyaJirwwAAAICTMVYHAAAAnK9AhfRnn31Wzz33nFq1aqVWrVo5OyYAAAAABcRYHQAAAHC+At1sdPfu3cxkAQAAAIohxuoAAACA8xWokN66dWutXbtWGRkZzo4HAAAAwA1grA4AAAA4X4GWdvH19dXatWv1xRdfKCwsTP7+/lbbTSaT5s+f75QAAQAAANiPsToAAADgfAUqpCclJSkiIsL43WKxWG2/+ncAAAAARYOxOgAAAOB8dhfSv/zyS91xxx0qU6aMPv3008KMCQAAAIADGKsDAAAAhcvuNdL/85//6NixY1aPzZkzR2fPnnV2TChmzOZMxcfH68iRQ3b/ZGZmujpsAACAWwZjdQAAAKBw2T0j/epLQM1ms9555x3deeedKl++vNMDQ/GRmnRKSRnpii9lX3E8JfGEukiqWbNWocYFAACAbIzVAQAAgMJVoDXSc7C+4q0joHKwKoeGuToMAAAA2ImxOgAAAOA8di/tAgAAAAAAAADArYhCOgAAAAAAAAAANtxwId1kMjkjDgAAAABOxlgdAAAAcA6H1kgfNGiQfHx8rB4bMGCAvL29rR4zmUzasmXLjUcHAAAAwC6M1QEAAIDCY3ch/dFHHy3MOHATMZszFR9/yuH9qlULlZfXDd3/FgAA4JbEWB0AAAAoXHZXLSdOnFiYceAmkpp0SkkZ6YovlWn3PimJJ9RFUs2atQotLgAAgJsVY3UAAACgcDH9F4UioHKwKoeGuToMAAAAAAAAALhhN3yzUQAAAAAAAAAAbmbMSAcAAAAAFJpMs1kJ8fEO7cP9k25+mZmZio8/Zvzu7e2pjAyzzX14XwAAXIn/gQAAAAAAhebUuVRlfbtVnkHBdrU/kXpW6tqD+yfd5OLjjylxxRJVDSgvmSSZTPK0WCRL/u15XwAAXI1COgAAAACgUAUFBCiscmW729uel3zzuXp2tj1uhtnZVQPKZ78vTJKHyaQsG4V06dZ7XwAAihf3/l8XAAAAAAA3ZzU72w7MzgYAoOhRSAcAAAAAwMWM2dl2YnY2AABFy60K6f369VNAQIAmTZokSdq/f7/GjBmjP//8U7Vq1dIbb7yhhg0bGu3Xr1+vadOmKTk5WXfddZfGjRungIAAV4UPAAAAAHCy3Mui2HPDSunmWBYFAAAULQ9XB2CvDRs2aPv27cbvly5dUr9+/RQZGamVK1cqIiJC/fv316VLlyRJ+/btU3R0tAYPHqwlS5bo4sWLGjFihKvCBwAAAAAUgpxlUTy3bZG2fCnPbVvkufXaP4krlji8HjkAAIBbfAV//vx5TZkyRY0aNTIe27hxo3x9fTVs2DCZTCZFR0drx44d2rRpk6KiorRw4UJ16tRJXbp0kSRNmTJFbdu2VUJCgkJCQlzUEwAAAACAs+Usi2LPDSsllkUBAACOc4tC+uTJk/XII4/ozJkzxmNxcXFq1qyZTCaTJMlkMqlp06bau3evoqKiFBcXp3//+99G+ypVqigoKEhxcXEU0gEAAAAAt4zcy9/Yy5Hlbwry/PHx8QpVlkP7AADgSsW+kL5r1y7973//07p16/T6668bjycnJ6tWLes7lJcvX14HDx6UJJ05c0aVKlXKsz0pKanQYwYAAAAAoLjIWf6makB5u9qfSD0rde2hmjVrXb9xAZ5fkpIPH1Tl4Kp2twcAwNWKdSH9ypUrGjNmjEaPHq0SJUpYbUtLS5OPj4/VYz4+PkpPT5ckXb582eZ2e/3fhPcilXNMk657ReJNxxWvtzMYOXPT+G9F5Mz9kDP3Q87cDzkDgJtXzvI39nJ0+RtHnz8+JdnBIwAA4FrFupA+Y8YMNWzYUK1bt86zzdfXN09RPD093Si4X2u7n5+f3cf38fEsQNQ3ztvbUyaTWSaP7GL69Zg8spe28bDz1rHFrX3OPt7envL2ds1rfqNMJsnT01Mmk2S51b79cFPkzP2QM/dDztwPOQMAAACA/BXrQvqGDRuUkpKiiIgISTIK45s3b1bnzp2VkpJi1T4lJcVYziUwMDDf7RUrVrT7+OnpZpfMyMrIMMtisciSZd+MdEuWZLFYlGXn8nLFrX3OPhkZZmVkuOdtf3IKDpmZZgoPboKcuR9y5n7ImfshZwAAAACQv2JdSP/000+VmZlp/P7WW29Jkl5++WX9/PPPmjNnjiwWi0wmkywWi3bv3q0BAwZIksLDwxUbG6uoqChJUmJiohITExUeHu5QDK44icw55q14/uruJ+0Wi/v34VZDztwPOXM/5Mz9kDMAAAAAsFasC+nBwcFWv5csWVKSVL16dZUvX15vv/22JkyYoJ49e2rx4sVKS0tTp06dJEm9evVS79691aRJEzVq1EgTJkxQmzZtFBISUuT9AAAAAAAAAAC4r2JdSLelVKlSmjVrlsaMGaOlS5eqTp06mj17tvz9/SVJERERGjt2rKZPn64LFy6oVatWGjdunIujBgAAANzH8ePHNXbsWO3evVtly5bVk08+qeeee06SlJCQoNdee0179+5VUFCQRo4cqbvuusvY9/vvv9ebb76phIQEhYeHa8KECUxqwS0jMzNT8fHH7G4fHx+vUDmwNiYAAChyblVInzRpktXvjRs31qpVq67ZPioqyljaBQAAAID9srKy1K9fPzVq1EirVq3S8ePHNXToUAUGBqpz584aNGiQateurRUrVmjLli0aPHiwNm7cqKCgIJ06dUqDBg3SkCFD1Lp1a8XExGjgwIFau3atTK64CRFQxOLjjylxxRJVDShvV/vkwwdVObhqIUcFAABuhFsV0gEAAAAUjZSUFNWrV0+vv/66SpUqpdDQULVs2VKxsbGqUKGCEhIStHjxYvn7+yssLEy7du3SihUrNGTIEC1btkwNGzZU3759JUkTJ05Uq1at9NNPP6lFixYu7hlQNKoGlFdY5cp2tY1PSS7kaAAAwI2ikA4AAAAgj0qVKmnatGmSJIvFot27d+vnn3/WmDFjFBcXp/r16xvLKkpSs2bNtHfvXklSXFycIiMjjW1+fn5q0KCB9u7dSyEdQLHk6HI8klStWqi8vCirAMCtgk98AAAAADa1a9dOp06dUtu2bdWhQwe9+eabqlSpklWb8uXLKykpSZKUnJxsc7u9inoVGKvj2Xts01V/0t5px3Ao/1e3tbVvQZ6/IEzXiePqtrn/tLO9w69RcXn+XMewN3eZWWYlxMc7cACpenX7C93x8ceUuNL+5XhOpJ6VonooLKyWQzHdjExF9W8KTkfu3Be5cw0K6QAAAABsmj59ulJSUvT6669r4sSJSktLk4+Pj1UbHx8fpaenS9J1t9vDx8fzxgN3kLe3p2SSPBw4K/UwmWTysH+fW619QY/h4e2ZnQ87ZOfNlH0ck+RxnWquo89fELljskdRvkbF4flzjpG7z9fLXdL5c8r6bpu8jwbb9fwnzp6Vd49edhe6vb09Va18BbuX4/EwmaRCfh+5C5NJ8vT0lMkkWSyujgaOIHfui9y5BoV0AAAAADY1atRIknTlyhW9/PLL6tq1q9LS0qzapKenq0SJEpIkX1/fPEXz9PR0lSlTxu5jpqebi3yWVUaGWV4WKcuBM9Isi0WWLPv3udXaF2Sf9MxMJRw+qowMs13t4+PjFZplVpbFIg+ZrnucLItF5gyz3c9fEBkZZnlaLIWaB0f6UJB4CvP5c46Ru8/Xy12WxaKgcrepRmCg3c+fUYxeo5tZTiEvM9NMQc/NkDv3Re5cg0I6AAAAgDxSUlK0d+9etW/f3nisVq1aysjIUMWKFXXkyJE87XOWcwkMDFRKSkqe7fXq1XMohqI+MbQ6nr3Htlz1J+1veJ9TqanK+narPIPsm3mcfPigKgdXzf+YNuIp9PeX5TpxXN029592tneoDwWIp9CeP9cxZJH1ci7Xeg4HX6PMTMeWgomPj1eoJatwX6ObnMXC6+GuyJ37IndFi0I6AAAAgDxOnDihwYMHa/v27Qr8vxmgv/76qwICAtSsWTN9/PHHunz5sjELPTY2Vs2aNZMkhYeHKzY21niutLQ07d+/X4MHDy76jsAtBQUE2L3ERnxKciFHA3d06pwTvpABACAXCukAAAAA8mjUqJEaNGigkSNHasSIETp58qSmTp2qAQMGqHnz5qpSpYpGjBihgQMHauvWrdq3b58mTpwoSeratavmzp2r2bNnq23btoqJiVHVqlXVokULF/cKuDlkmgsw21pZhRhR8cQXMgAAZ6KQDgAAACAPT09PffDBBxo3bpx69OghPz8/9e7dW3369JHJZNIHH3yg6OhoRUVFqXr16oqJiVFQUJAkqWrVqnr//ff15ptvKiYmRhEREYqJiZGpqBc9B/LhaBFakqpVC5WXV/E5fWa2NQAARa/4jAQAoIhlZmbq8OHDDu1T3E6iAAAoTIGBgZoxY0a+26pXr66FCxdec9977rlH99xzT2GFBhSYo0Xo4ynJim/RStWqVbP7GEUxA5zZ1gAAFC2qQQBuWcePH9Pq346pQhX7ZuekJJ5QF0k1a9Yq1LgAAABQuBwtQjtSeJeYAQ4AwM2IQjqAW1qFKlVVOTTM1WEAAACgGHOk8C4xAxwAgJuRh6sDAAAAAAAAAACgOKOQDgAAAAAAAACADSztAgAAAAAA4IBMs1kJ8fEO71etWqi8vCjFAIA74tMbAAAAAADAAafOpTp8E9oTqWelrj1Us2atQowMAFBYKKQDAAAAAAA4yNGb0EqSuZBiAQAUPtZIBwAAAAAAAADABgrpAAAAAAAAAADYQCEdAAAAAAAAAAAbWCMdwE0jMzNT8fHH7G6fmHhCFt9KhRcQAAAAAAAAbgoU0lEsmM2Zio8/5dA+1aqFysuLtzD+v/j4Y1r92zFVqFLVrvaHDsQruE7ZQo4KAAAAAAAA7o4qJIqF1KRTSspIV3ypTLvapySeUBdJNWvWKtS44H4qVKmqyqFhdrVNSYwv5GgAAAAA95NpNish3v6xcnx8vEKVVYgRAQDgehTSUWwEVA62uwAKAAAAACgcp86lKuvbrfIMCrarffLhg6ocbN9VoQAAuCsK6QAAAAAAwEpQQIDCKle2q218SnIhR3NzcHSmv8SSpgBQnPBpDAAAAAAAUMgcnel/IvWs1LUHS5oCQDFBIR0AAAAAAKAIODLTX5LMhRgLAMAxHq4OAAAAAAAAAACA4owZ6QAKJDMzU/Hxxxzah/X9AAAAAAAA4I6oaAEokPj4Y1r92zFVqFLVrvYpiSfURWJ9PwAAAAAAALgdCukACqxClaqqHBrm6jAAAAAAAACAQsUa6QAAAAAAAAAA2MCMdLglszlT8fGnHNqH9bkBAAAAAAAAFARVRbil1KRTSspIV3ypTLvasz43AAAAAAAAgIKikA63FVA5mPW5AQAAAAAAABQ6CukAJEmZmZmKjz9md/v4+HhZSlUuvICKIZYUAgAAAAAAuDVR3QEgSYqPP6bVvx1ThSpV7Wr/56ETCql7WyFHVbywpBAAAAAAAMCtiUI6bgnMJLZPhSpV7V4uJ/lkfCFHUzyxpBAAAAAAAMCt59aqEuKWxUxi1yvIlxnSrfmFBgAAAAAAAIoXqlO4ZTCT2LUc/TJDcv8vNPjyAAAAAAAA4OZApQZAkXH0y4zsQrT9S8gUtxug3opfHgAAAAAAANyMKKQDKLYcLUQXxxugciUEAAAAAACA+6OQDqBYc6QQfaveABUAAAAAAACFy8PVAQAAAAAAAAAAUJwxIx0AipGC3KCUm5MCAAAAAAAULiovAFCMOLouPDcnBQAAAAAAKHwU0gEXyczMVHz8MYf2YebxrYEblAIAAAAAABQvVOQAF4mPP6bVvx1ThSpV7WrPzGMAAAAAAADANSikAy5UoUpVZh4DAAAAAAAAxZyHqwMAAAAAAAAAAKA4o5AOAAAAAAAAAIANLO0CAAAAAABQzGSazUqIj3don2rVQuXlRakHAAoDn64AAAAAAADFzKlzqcr6dqs8g4Ltan8i9azUtYdq1qxVyJEBwK2JQjoAuDGzOVPx8acc2odZKgAAAIB7CAoIUFjlyna3NxdiLABwq6OSAgBuLDXplJIy0hVfKtOu9imJJ9RFYpYKAAAAAACAAyikA4CbC6gcrMqhYa4OAwAAAAAA4KZFIR1wEyzhAQAAAAAAALgGFTbACTIzMxUff1QZGfavSBcfHy9LKfvXunN0CY8zJ44rMj5e1apVK5R4AAAAAAAAgFsFhXTACY4fP6a1vx9TQGBVu/f589AJhdS9zaHjOLKER/LJeO1KuWR34b0g8QAAAAAAAAC3AgrpgJOUr1JVgdXsX6c6+WR8IUaTzdHCOwAAAAAAAIC8PFwdAAAAAIDi5/Tp03rhhRfUvHlztW7dWhMnTtSVK1ckSQkJCXr66afVpEkTPfDAA9q5c6fVvt9//706d+6s8PBw9enTRwkJCa7oAgAAAOA0zEgH8uHojT3j4+OlUlUKMSIAAICiY7FY9MILL6hMmTL67LPPdOHCBY0cOVIeHh4aNmyYBg0apNq1a2vFihXasmWLBg8erI0bNyooKEinTp3SoEGDNGTIELVu3VoxMTEaOHCg1q5dK5PJ5OquAQAAAAVCIR3Ih6M39vzz0AlVqxdQyFEBAAAUjSNHjmjv3r367rvvVKFCBUnSCy+8oMmTJ+vuu+9WQkKCFi9eLH9/f4WFhWnXrl1asWKFhgwZomXLlqlhw4bq27evJGnixIlq1aqVfvrpJ7Vo0cKV3QIAAAAKjEI6cA2sLw4AAG5VFStW1EcffWQU0XP8/fffiouLU/369eXv72883qxZM+3du1eSFBcXp8jISGObn5+fGjRooL1791JIBwAAgNtijXQAAAAAVsqUKaPWrVsbv2dlZWnhwoW64447lJycrEqVKlm1L1++vJKSkiTputsBAAAAd8SMdAAAAAA2TZ06Vfv379fy5cs1b948+fj4WG338fFRenq6JCktLc3mdkcU9ZLqVsez99imq/6kfdEfI792tva9mfrsru1t7XOt5yhufSjG7wtXfXZyGwz3Q+7cF7lzDQrpAAAAAK5p6tSpmj9/vt59913Vrl1bvr6+On/+vFWb9PR0lShRQpLk6+ubp2ienp6uMmXKOHRcHx/PG4q7ILy9PSWT5OHAWamHySSTh/373Grtizomk0nyuE7V8Wbrszu2z2+f6+WuuPWhuL4vPLw9sz/LipDJJHl6espkkiyWIj00bhC5c1/kzjUopAMAAADI17hx47Ro0SJNnTpVHTp0kCQFBgbq0KFDVu1SUlKM5VwCAwOVkpKSZ3u9evUcOnZ6urnIZ1llZJjlZZGyHDgjzbJYZMmyf59brX1Rx+Qh03X3u9n67I7t89vnerkrbn0oru8Lc4ZZGRlmu2NyhpxCXmammYKemyF37ovcuQaFdAAAAAB5zJgxQ4sXL9Y777yjjh07Go+Hh4dr9uzZunz5sjELPTY2Vs2aNTO2x8bGGu3T0tK0f/9+DR482OEYivrE0Op49h7bctWftC/6Y+TXzta+N1Of3bX91fuY8nm8qGO6Cd4XmZlmJcTHOxCQVK1aqLy8nFMasliYGeuuyJ37IndFi0I6AAAAACuHDx/WBx98oH79+qlZs2ZKTk42tjVv3lxVqlTRiBEjNHDgQG3dulX79u3TxIkTJUldu3bV3LlzNXv2bLVt21YxMTGqWrWqWrRo4aruAMAt4dS5VGV9u1WeQcF2tT+Relbq2kM1a9Yq5MgA4OZAIR0AAACAla+//lpms1kzZ87UzJkzrbb98ccf+uCDDxQdHa2oqChVr15dMTExCgoKkiRVrVpV77//vt58803FxMQoIiJCMTExMnE3LAAodEEBAQqrXNnu9kW7CAwAuDcK6QAAAACs9OvXT/369bvm9urVq2vhwoXX3H7PPffonnvuKYzQAAAAAJfwcHUAAAAAAAAAAAAUZ8xIB4BbiNmcqfj4Uw7t48wbEAEAAAAAALgjKiMAcAtJTTqlpIx0xZfKtKt9SuIJdZG4AREAAAAAALilUUgHgFtMQOVgVQ4Nc3UYAAAAAAAAboM10gEAAAAAAAAAsIFCOgAAAAAAAAAANlBIBwAAAAAAAADABgrpAAAAAAAAAADYQCEdAAAAAAAAAAAbvFwdAADg5pGZman4+GMO71etWqi8vPgvCQAAAAAAFE9ULQAAThMff0yrfzumClWq2r1PSuIJdZFUs2atQosLAAAAAADgRlBIBwA4VYUqVVU5NMzVYQAAAAAAADgNhXQAAAAAAIBbTKbZrIT4eIf2YUlGALcyPv0AAAAAAABuMafOpSrr263yDAq2q/2J1LNS1x4syQjglkUhHQAAAAAA4BYUFBCgsMqV7W5vLsRYAKC4K/aF9NOnT2vChAn64Ycf5OvrqwceeEBDhw6Vr6+vEhIS9Nprr2nv3r0KCgrSyJEjdddddxn7fv/993rzzTeVkJCg8PBwTZgwQSEhIS7sDQC4F7M5U/Hxp+xuHx8fL0sp+wfiAAAAAAAA7qBYF9ItFoteeOEFlSlTRp999pkuXLigkSNHysPDQ8OGDdOgQYNUu3ZtrVixQlu2bNHgwYO1ceNGBQUF6dSpUxo0aJCGDBmi1q1bKyYmRgMHDtTatWtlMplc3TUAcAupSaeUlJGu+FKZdrX/89AJhdS9rZCjAgAAAAAAKFrFupB+5MgR7d27V999950qVKggSXrhhRc0efJk3X333UpISNDixYvl7++vsLAw7dq1SytWrNCQIUO0bNkyNWzYUH379pUkTZw4Ua1atdJPP/2kFi1auLJbAOBWAioHq3JomF1tk086drMiAAAAAAAAd+Dh6gBsqVixoj766COjiJ7j77//VlxcnOrXry9/f3/j8WbNmmnv3r2SpLi4OEVGRhrb/Pz81KBBA2M7AAAAAAAAAAD2KNaF9DJlyqh169bG71lZWVq4cKHuuOMOJScnq1KlSlbty5cvr6SkJEm67nYAAAAAAAAAAOxRrJd2udrUqVO1f/9+LV++XPPmzZOPj4/Vdh8fH6Wnp0uS0tLSbG63lyuWU885pkmSpegPjwIw5fqTnLkH7pRQ/Fzv89b4bCR5boOcuR9yBgAAriXTbFZCfP5LOXp7eyojw5zvtmrVQuXl5VblJwDIl9t8kk2dOlXz58/Xu+++q9q1a8vX11fnz5+3apOenq4SJUpIknx9ffMUzdPT01WmTBm7j+nj43nDcReEt7enTCazTB72FftMHpLJZJKHndcXFLf2xTEmh/vgkV10sDdnRRGTu7cvkphMJnl43GJ9Lmbtc/bx9vaUt7ftz1yTSfL09JTJJFn4xsotkDP3Q84AAMC1nDqXqqxvt8ozKNh6g0mSySRPiyXPzLITqWelrj1Us2atIosTAAqLWxTSx40bp0WLFmnq1Knq0KGDJCkwMFCHDh2yapeSkmIs5xIYGKiUlJQ82+vVq2f3cdPTzS6ZkZWRYZbFYpEly77ZzZYsyWKxKCvLvucvbu2LY0wO9yEru+Bgb86KIiZ3b1/YxzApp+2t0+fi2D5nn4wM8zVnsOTIKexlZpop8LkJcuZ+yBkAALAlKCBAYZUrWz9okjxMJmXlU0iXJNujfABwH8W+kD5jxgwtXrxY77zzjjp27Gg8Hh4ertmzZ+vy5cvGLPTY2Fg1a9bM2B4bG2u0T0tL0/79+zV48GCHju+Kk8icY3L+6j4sV/2J4o9cFT/2ft5aLMyUdTfkzP2QMwAAAACwVqxvNnr48GF98MEH+ve///3/2rv7+Jrr/4/jz7MNmyQxjZH84rdZzDbTKGOR8GPki/pKQvr+KBNduKyvKLmIXIQR5aLrfKVcdfFL35R8i/qu6FuqoXI2i21mEbve+/eHr8/XMY5zXGznnD3ut9tudd4Xn8/783mfzevzun0+749iY2OVlZVl/cTFxal+/fqaOHGi9uzZo2XLlunbb79Vv379JEl9+/bV119/rWXLlmnPnj2aOHGiGjZsqDZt2lTwUQEAAAAAAAAAvIlHJ9L//ve/q6SkREuWLFF8fLzDj7+/vxYvXqysrCz16dNHGzZsUHJyskJDQyVJDRs21MKFC7V27Vr169dPubm5Sk5Olo23ZwEAAAAAAAAA3ODRS7sMGzZMw4YNO2f9ddddp1dfffWc9QkJCUpISLgcQwMAAAAAAAAAVBIefUc6AAAAAAAAAAAVjUQ6AAAAAAAAAABOkEgHAAAAAAAAAMAJEukAAAAAAAAAADhBIh0AAAAAAAAAACdIpAMAAAAAAAAA4ASJdAAAAAAAAAAAnCCRDgAAAAAAAACAEyTSAQAAAAAAAABwgkQ6AAAAAAAAAABOBFT0AAAAlVtJSbHs9gyX2lap4q+iohI1atRYAQH8EwYAAAAAAMoHWQgAQIXKOZihg0WFstcoPm9bm1+xsg6kq7ek669vetnHBgAAAAAAIJFIBwB4gNr1Gqhe4ybnbefnJ5nSchgQAAAAAADAaVgjHQAAAAAAAAAAJ0ikAwAAAAAAAADgBIl0AAAAAAAAAACcIJEOAAAAAAAAAIATJNIBAAAAAAAAAHAioKIHAAAAAAAAAN9TXFKiNLvdrT6NGjVWQADpKgCeh79MAAAAAAAAuOQyjuSo9LMt8g9t4FL79JzDUt8/6/rrm17mkQGA+0ikAwAAAAAA4LIIrV1bTerVc7l9yWUcCwBcDBLpAAAAAAAAqHAsBQPAk/GXBgAAAAAAABWOpWAAeDIS6QAAAAAAAPAILAUDwFP5VfQAAAAAAHiuwsJCJSYmaseOHVZZWlqahgwZoujoaHXv3l3btm1z6PP5558rMTFRUVFRGjRokNLS0sp72AAAAMAlRSIdAAAAwFkVFBTokUce0Z49e6wyY4ySkpIUHBystWvX6vbbb9fIkSOVkZEhScrIyFBSUpL69Omjt956S7Vr19aIESNkjKmowwAAAAAuGol0AAAAAGXs3btXd955p+xnvPRt+/btSktL01NPPaUmTZpo+PDhio6O1tq1ayVJa9asUYsWLTR06FD993//t2bMmKEDBw7oyy+/rIjDAAAAAC4JEukAAAAAyvjyyy/Vpk0brV692qF8165duuGGG1S9enWrLDY2Vjt37rTqW7dubdUFBQWpefPmVj0AAADgjXjZKADAq5SUFMtuz3CrT6NGjRUQwD95AOCOAQMGnLU8KytL11xzjUNZnTp1dPDgQZfqAQAAAG9EVgEA4FVyDmboYFGh7DWKXWqf/Vu6eku6/vqml3VcAFBZ5OXlqWrVqg5lVatWVWFhoUv17rDZLnycF8Jhf67u23bGf2lf/vs4WztnfX3pmL21vbM+59qGpx0D3wvnbcp5TOX974WvsHH+vBZzVzFIpAMAvE7teg1Ur3GTih4GAFRK1apVU25urkNZYWGhAgMDrfozk+aFhYWqWbOmW/upWtX/osZ5IapU8Zdskp8bV6V+Nptsfq73qWzty3tMNpvkd54MnK8dsze2P1uf882dpx0D3wtH55q/8hiTXxX/k3+/4TabTfL395fNJvFOcO/C3FUMEukAAJ/GUjAAcGmFhIRo7969DmXZ2dnWci4hISHKzs4uUx8REeHWfgoLS8r9LquiohIFGKnUjSvSUmNkSl3vU9nal/eY/GQ7bz9fO2ZvbH+2PuebO087Br4Xjs41f+UxppKiEhUVlbjUHo5OJWGLi0tIxnoZ5q5ikCUAAPg0loIBgEsrKipKy5YtU35+vnUXekpKimJjY636lJQUq31eXp52796tkSNHur2v8r4wdNifq/s2Z/yX9uW/j7O1c9bXl47ZW9uf2cd2lvLyHhPfiwtv72z+ymlMJBIvjjGcQ2/F3JUvEukAAJ/HUjAAcOnExcWpfv36mjhxokaMGKEtW7bo22+/1YwZMyRJffv21fLly7Vs2TJ17NhRycnJatiwodq0aVPBIwcA+JrikhKl2e1u9eHpUwAXir8cAAAAAFzm7++vxYsX6/HHH1efPn103XXXKTk5WaGhoZKkhg0bauHChZo+fbqSk5MVExOj5ORk2XgbFgDgEss4kqPSz7bIP7SBS+3Tcw5Lff/M06cALgiJdAAAAABO/fTTTw6fr7vuOr366qvnbJ+QkKCEhITLPSwAABRau7aa1KvncntWUwdwofwqegAAAAAAAAAAAHgyEukAAAAAAAAAADhBIh0AAAAAAAAAACdIpAMAAAAAAAAA4ASJdAAAAAAAAAAAnCCRDgAAAAAAAACAEyTSAQAAAAAAAABwIqCiBwAAAAAAAABcbsUlJUqz293q06hRYwUEkD4DQCIdAAAAAAAAlUDGkRyVfrZF/qENXGqfnnNY6vtnXX9908s8MgDegEQ6AAAAAAAAKoXQ2rXVpF49l9uXXMaxAPAurJEOAAAAAAAAAIATJNIBAAAAAAAAAHCCpV0AADhNSUmx7PYMt/rwAiIAAAAAAHwbV/0AAJwm52CGDhYVyl6j2KX22b+lq7fEC4gAAAAAAPBhJNIBADhD7XoNVK9xE5fanryD3e7W9rmDHQAAAAAA78JVPAAAF4E72AEAAAAA8H0k0gEAuEiX+w52ibvYAQAAAACoSFyRAwBQjty9g12SMtP3q7XdrkaNGrnch8Q7AAAAAACXDlfYAACUM3fuYJekrAN2fZF9guVjAAAAAACoICTSAQDwAu4m3wEAAAAAwKVDIh0AAAAAAAA4Q3FJidLcfL8RSywCvovfbAAAAAAAAOAMGUdyVPrZFvmHNnCpfXrOYanvn1liEfBRJNIBAAAAAACAswitXVtN6tVzuX3JZRwLgIrlV9EDAAAAAAAAAADAk3FHOgAAAAAAAHCRLmRNdYl11QFvwW8pAAAAAAAAcJHcXVNdkvZnZ8nepp0aNWrkch8S70DF4LcOAAAAAAAAuATcXVPdnp3FC00BL0EiHQAAAAAAAKggvNAU8A68bBQAAAAAAAAAACe4Ix0AAAAAAADwAhfyQlPWVAcuDX6LAAAAAAAAAC/g7gtNWVMduHRIpAMAAAAAAABegjXVgYrBGukAAAAAAAAAADhBIh0AAAAAAAAAACdIpAMAAAAAAAAA4ASJdAAAAAAAAAAAnOBlowAAAAAAAIAPKi4pUZrdfs76KlX8VVTk+DrSRo0aKyCAlCFwJn4rAAAAAAAAAB+UcSRHpZ9tkX9og7KVNkk2m/yNkczJov3ZWbK3aadGjRq5vA8S76gs+JYDAOBjSkqKZbdnuNWH4BcAAADwTaG1a6tJvXplK2ySn82m0tMS6fbsrHMn3s8iPeew1PfPuv76ppdwxIBn4ooZAAAfk3MwQweLCmWvUexS++zf0tVbIvgFAAAAcO7E+1kUl5TI7mTpmHPhRh54I76xAAD4oNr1Gqhe4yaXZdvFxcWy2391qe2pNRcJlAEAAADf43TpmHPgLnZ4K65oAQCo5NxdCsZut+vrYyUKrn/tedva/Ip1yL5fre12t9ZZlLhLBQAAAPAG7tzBfkrJ+ZsAHoerUwAAKjl3l4JJ3Zuua5tFunTHu5+flJlm1xfZJ1zevsRyMwAAAAAAz0IiHQAAuLUUTNYB99dAvJxLzQAAAAAAcLmRSAcAAAAAAABQLopLSpTmxgtKi4uLJdkUEOB/WdpLLCsJ1/ANAQAAHsfdddslgl8AAADAG7j7gtKUfXtUJ6i6mlym9rz8FK7iahMAAHgcd9dtZ011AAAAwHu484JSe3aW6taocdnaS7z8FK4hkQ4AADwS66oDAAAAADwFiXQAAOD1WAoGAAAAAHA5cfUIAAC8HkvBAAAAALgQl/vlpxI38fgKZhAAAPiEy7kUTHFxsez2X93qQ7AMAAAAeL7L/fJTXmbqO3z66q6goEBPPvmkPvzwQwUGBmro0KEaOnRoRQ8LAABUMHeXgrHb7fr6WImC61/rUnvueAeIxQEAgPe4nC8/lXiZqa/w6UT6rFmz9N133+mll15SRkaGxo8fr9DQUHXr1q2ihwYAACqQu0vBpO5N17XNIl2+4/1kot71x0Ml7mCH7yEWBwAAgC/x2au1EydOaM2aNXrhhRfUvHlzNW/eXHv27NFrr71G8A4AANxaCibrgHtJcXcT9Znp+9XablejRo1c3oe7azO62r5KFX8VFZ28Z4bkPi4UsTgAAMBJ7q7BLrkWu58et7Nue/nw2bP1448/qri4WDExMVZZbGysnn/+eZWWlsrPz68CRwcAAHydu4n6L7JPuJx4l6TUnV/qilq11aCxa8vHuNre5lcsU3phyX2CcZxCLA4AAHCSu2uwSy6sw26TZLPJ3xjJuL9u+/7sLNnbtCPWd5PPHn1WVpauvvpqVa1a1SoLDg5WQUGBcnNzVbt27QocHQAAgCN3X5aadcCuGnXqupWsd6W9n59UWup+ct/dxPuF3DVzue7CPx0XCJcGsTgAAMB/uLMGu+TCOuw2yc9mU+m/E+nurttuz85yK7nv7gtTi4uLZbf/6lLb03l6LO65I7tIeXl5DoG7JOtzYWGhy9ux2S7psFze5+Hf0qVSybjQ/kjWQRUWFSooMMil7Xtae08ck7vtc7MOqqi4SEHVAl2as/IYk7e3v9z7sEnKzTyogoLKc8ye2N6dPjZJ8jv5++ZJx+BJ58jT2l/onF3OMflK+8u1j1NzptKT7a+o5XqyMzf7kN5Nz1fdw/kutbf/9J2CatZS3foNXd6Hu33cbf/74Uzd11lq0oSXxF4sb43FbTYp/fDhkxelLjr4e66KCgsVFOTa72Jla1/eY7KSChU4nvLYh7e3P1uf882dpx0D3wtH55o/bzqGimjvCWM6c+4qwzF7entX+5w+dxcypjpB1f99AeACm9x6B5Tdbteh/9uka2pe5XKfzKO/S/87wqNjcZsxbkSJXuT999/X008/rX/84x9W2b59+9S9e3ft2LFDtWrVqrjBAQAAAD6MWBwAAAC+xmcXJwwJCdGRI0f+/UjvSVlZWQoMDFTNmjUrcGQAAACAbyMWBwAAgK/x2UR6RESEAgICtHPnTqssJSVFkZGRvNwIAAAAuIyIxQEAAOBrfDaKDQoKUu/evTVlyhR9++23+uijj7RixQoNGjSooocGAAAA+DRicQAAAPgan10jXTr5kqMpU6boww8/VI0aNXTfffdpyJAhFT0sAAAAwOcRiwMAAMCX+HQiHQAAAAAAAACAi+WzS7sAAAAAAAAAAHApkEgHAAAAAAAAAMAJEukAAAAAAAAAADhBIt2DFBQU6LHHHlPr1q0VHx+vFStWVPSQKrXCwkIlJiZqx44dVllaWpqGDBmi6Ohode/eXdu2bXPo8/nnnysxMVFRUVEaNGiQ0tLSHOpXrVql9u3bKyYmRo899pjy8vLK5Vh83aFDhzRq1CjFxcWpffv2mjFjhgoKCiQxZ55q//79uu+++xQTE6NbbrlFL774olXHnHm2YcOGacKECdbn3bt364477lBUVJT69u2r7777zqH9pk2b1LlzZ0VFRSkpKUk5OTlWnTFGzz77rNq2bau4uDjNmjVLpaWl5XYsvm7z5s0KDw93+Bk1apQk5g04G2Jxz0RM7n2Izb0bcbpvIGb3PsTuXsLAYzz11FOmZ8+e5rvvvjMffvihiYmJMe+//35FD6tSys/PN0lJSSYsLMxs377dGGNMaWmp6dmzp3n00UfN3r17zfPPP2+ioqLMgQMHjDHGHDhwwERHR5vly5eb1NRUM3r0aJOYmGhKS0uNMcZ88MEHJjY21nz88cdm165dpnv37ubJJ5+ssGP0FaWlpebOO+80f/nLX0xqaqr56quvzG233WZmzpzJnHmokpIS06VLF/Poo4+aX375xXzyySemVatWZsOGDcyZh9u0aZMJCwsz48ePN8YYc/z4cdOuXTszc+ZMs3fvXjN16lRz8803m+PHjxtjjNm1a5dp2bKleeedd8wPP/xgBg4caIYNG2Ztb/ny5SYhIcF89dVX5osvvjDx8fHmxRdfrJBj80WLFy82w4cPN5mZmdbP77//zrwB50As7nmIyb0Psbl3I073DcTs3onY3TuQSPcQx48fN5GRkVaAaIwxycnJZuDAgRU4qsppz549plevXqZnz54OQfvnn39uoqOjrT9WxhgzePBgs2DBAmOMMfPnz3eYrxMnTpiYmBir/4ABA6y2xhjz1VdfmZYtW5oTJ06Ux2H5rL1795qwsDCTlZVllW3cuNHEx8czZx7q0KFDZvTo0ebYsWNWWVJSkpk8eTJz5sGOHDliOnToYPr27WsF5WvWrDGdOnWyLpBKS0vNbbfdZtauXWuMMWbs2LFWW2OMycjIMOHh4cZutxtjjElISLDaGmPMunXrTMeOHcvrkHzeo48+aubMmVOmnHkDyiIW9zzE5N6J2Ny7Ead7P2J270Xs7h1Y2sVD/PjjjyouLlZMTIxVFhsbq127dvHIRTn78ssv1aZNG61evdqhfNeuXbrhhhtUvXp1qyw2NlY7d+606lu3bm3VBQUFqXnz5tq5c6dKSkr0r3/9y6E+OjpaRUVF+vHHHy/vAfm4unXr6sUXX1RwcLBD+R9//MGceahrrrlG8+fPV40aNWSMUUpKir766ivFxcUxZx7smWee0e23366mTZtaZbt27VJsbKxsNpskyWazqVWrVuecr/r16ys0NFS7du3SoUOH9Ntvv+nGG2+06mNjY3XgwAFlZmaWz0H5uH379qlx48Zlypk3oCxicc9DTO6diM29G3G69yNm917E7t6BRLqHyMrK0tVXX62qVataZcHBwSooKFBubm7FDawSGjBggB577DEFBQU5lGdlZemaa65xKKtTp44OHjx43vqjR4+qoKDAoT4gIEC1atWy+uPC1KxZU+3bt7c+l5aW6tVXX1Xbtm2ZMy/QqVMnDRgwQDExMeratStz5qG++OIL/fOf/9SIESMcys83X5mZmeesz8rKkiSH+lMX3czXxTPG6JdfftG2bdvUtWtXde7cWc8++6wKCwuZN+AsiMU9DzG5dyI29x3E6d6HmN17Ebt7j4CKHgBOysvLcwjcJVmfCwsLK2JIOMO55ujU/Dirz8/Ptz6fqz8ujdmzZ2v37t166623tGrVKubMwy1YsEDZ2dmaMmWKZsyYwe+ZByooKNDkyZP1xBNPKDAw0KHufPOVn5/v1nzx796lk5GRYc3P/PnzlZ6erqefflr5+fnMG3AWxOLeg1jBuxCbey/idO9CzO7diN29B4l0D1GtWrUyX+JTn8/8I4iKUa1atTJ3JBUWFlrzc645rFmzpqpVq2Z9PrP+zLtscOFmz56tl156SfPmzVNYWBhz5gUiIyMlnQz8xowZo759+yovL8+hDXNWsRYtWqQWLVo43F12yrnm43zzFRQU5BDAnTl3zNfFa9CggXbs2KGrrrpKNptNERERKi0t1dixYxUXF8e8AWcgFvcexHfeg9jcuxGnexdidu9G7O49WNrFQ4SEhOjIkSMqLi62yrKyshQYGKiaNWtW4MhwSkhIiLKzsx3KsrOzrUdkzlVft25d1apVS9WqVXOoLy4uVm5ururWrXv5B18JTJ06VStXrtTs2bPVtWtXScyZp8rOztZHH33kUNa0aVMVFRWpbt26zJmHeffdd/XRRx8pJiZGMTEx2rhxozZu3KiYmJiL+h0LCQmRJOtxw9P/n/m6NGrVqmWtpShJTZo0UUFBwUX9njFv8FXE4t6D+M47EJt7J+J070XM7v2I3b0DiXQPERERoYCAAOtlAZKUkpKiyMhI+fkxTZ4gKipK33//vfVojHRyjqKioqz6lJQUqy4vL0+7d+9WVFSU/Pz8FBkZ6VC/c+dOBQQEqFmzZuV3ED5q0aJFevPNNzV37lz16NHDKmfOPFN6erpGjhypQ4cOWWXfffedateurdjYWObMw7zyyivauHGj1q1bp3Xr1qlTp07q1KmT1q1bp6ioKH3zzTcyxkg6ubbf119/fc75+u233/Tbb78pKipKISEhCg0NdahPSUlRaGhomTX+4L7PPvtMbdq0cbhz7IcfflCtWrUUGxvLvAFnIBb3HsR3no/Y3HsRp3svYnbvRuzuRQw8xqRJk0yPHj3Mrl27zObNm02rVq3M//3f/1X0sCq1sLAws337dmOMMcXFxaZ79+7moYceMqmpqWbp0qUmOjraHDhwwBhjTFpamomMjDRLly41qampZvTo0aZnz56mtLTUGGPMpk2bTKtWrczmzZvNrl27TI8ePczUqVMr7Nh8xd69e01ERISZN2+eyczMdPhhzjxTcXGx6dOnjxk6dKjZs2eP+eSTT8zNN99sVq1axZx5gfHjx5vx48cbY4w5duyYadu2rZk6darZs2ePmTp1qmnXrp05fvy4McaYr7/+2jRv3tz87W9/Mz/88IMZOHCgGT58uLWtpUuXmvj4eLN9+3azfft2Ex8fb1asWFEhx+Vrjh07Ztq3b28eeeQRs2/fPvPJJ5+Y+Ph4s2zZMuYNOAdicc9FTO49iM29G3G67yBm9y7E7t6DRLoHOXHihBk3bpyJjo428fHxZuXKlRU9pErv9KDdGGN+/fVXc/fdd5sWLVqYHj16mH/84x8O7T/55BPTpUsX07JlSzN48GBjt9sd6pcuXWpuuukmExsbayZOnGjy8/PL5Th82dKlS01YWNhZf4xhzjzVwYMHTVJSkmnVqpVp166dWbJkiRVkM2ee7fSg3Bhjdu3aZXr37m0iIyNNv379zPfff+/Qfu3atSYhIcFER0ebpKQkk5OTY9UVFxeb6dOnm9atW5s2bdqY2bNnW98DXLzU1FQzZMgQEx0dbdq1a2cWLlxonV/mDSiLWNxzEZN7D2Jz70ec7huI2b0Psbt3sBnz72cDAAAAAAAAAABAGSz4BwAAAAAAAACAEyTSAQAAAAAAAABwgkQ6AAAAAAAAAABOkEgHAAAAAAAAAMAJEukAAAAAAAAAADhBIh0AAAAAAAAAACdIpAMAAAAAAAAA4ASJdABAuTHGVPQQvALnCQAAAJcS8aVrOE8AnCGRDsBj3XPPPQoPD3f4adasmVq1aqU+ffpo/fr1FT1Ej7Vw4UKFh4dX9DAc7NmzR3fddZdDWXh4uBYuXHjB27z//vu1Zs0al9q+/fbbCg8PV3p6+gXvrzykpKRo2LBh5bKvwsJCdevWTTt37iyX/QEAAO9AHH7hiMPLIg4vizgc8E4BFT0AAHDmhhtu0OTJk63PJSUlOnjwoFatWqVx48apVq1aSkhIqMARwlUffPCBvvnmm0u2vbfffluHDh1S3759L9k2PcGaNWu0b9++ctlX1apVNWbMGI0fP17r169XYGBguewXAAB4PuJw30Ec7hricADnwx3pADxajRo1FB0dbf3ExsaqR48eWrFihapUqaK33367ooeICpCfn69nn31W999/v/z8+KfsYnTu3FlVqlTRG2+8UdFDAQAAHoQ4HGdDHH7pEIcD3oe/egC8UrVq1VS1alXZbDarrLS0VMuWLdNtt92mFi1aqGvXrnrllVcc+tntdt1///1q06aNoqKi9Oc//1mffvqpVb9w4UJ16tRJW7ZsUbdu3RQVFaU777xTO3bscNhOZmamJk6cqISEBLVs2VL9+vXT3//+d4c24eHheu211/T4448rLi5OMTExGj16tLKzs10ejySlpqZq+PDhatWqlVq1aqWkpCSlpaVd9DmUpIyMDD3yyCOKi4tTVFSUBg8erN27d1v16enpCg8P1/vvv69Ro0YpJiZGcXFx+utf/6oTJ05Y7YqKivTss8+qQ4cOatmype677z6tW7fOeoRz4cKFWrRokXVeTn+M9I8//nA4R6NGjXI4R2ezdu1aFRQUqGPHjg7ln376qfr376/o6GjFx8friSee0NGjRx3a7Nq1S/3791dkZKRuueUWvfjiiw716enpGjdunOLj49W8eXPddNNNGjdunI4cOWK16dSpk6ZPn67BgwerZcuWevzxxyVJP/74o0aOHKm2bduqefPmat++vZ5++mnl5+dbfQsLCzV//nzdeuutatmypRITE/XOO+9IkiZMmKB33nlHBw4cUHh4uHWBWlBQoFmzZikhIUEtWrRQz5499d577zmM+1xjeumll9StWzdFRkaqffv2mjJliv744w+Hvj179tTKlStVWFjo9LwDAAAQhxOHE4cThwOVFYl0AB7NGKPi4mLrp6CgQD///LMmTpyo48eP6/bbb7faTpkyRQsWLFCvXr30/PPPq1u3bpo+fbqSk5MlnQzwhw8frry8PM2aNUuLFy9WrVq19MADD2j//v3WdnJycjR+/HgNGDBAzz33nAIDA3Xffffphx9+kCRlZ2erX79++uc//6mHH35YCxcuVIMGDZSUlKQNGzY4jH/evHkqLS3V3LlzNW7cOG3ZskXTp093eTy//PKL+vfvr8OHD+uZZ57RtGnTlJaWprvuukuHDx++qHObk5Oj/v376/vvv9ekSZM0Z84clZaW6u677y7zSOPkyZPVoEEDLV68WPfdd5/eeustLVmyxKp/4okn9NJLL2ngwIFKTk5WcHCwJk2aZNXfcccd6tevnyRp9erVuuOOO6y6l19+WUVFRXruuef06KOP6uOPP9ZTTz3ldOwbNmzQLbfcoqpVq1plW7Zs0fDhw1WnTh3Nnz9fY8aM0UcffaSHH37Yoe+UKVPUo0cPLVu2TDExMZo9e7a2bNkiScrLy9OgQYO0b98+TZ48WcuXL9egQYP07rvvat68eQ7bee211xQZGanFixerX79+yszM1N133628vDzNnDlTL7zwgnr06KFXXnlFL7/8stVvzJgxWrlype644w4tXbpU8fHxmjBhgjZt2qQRI0YoISFBdevW1erVq3XLLbfIGKOkpCS9+eabuvfee7VkyRLFxMTo4Ycf1rp165yOadOmTZo9e7buvvtuLV++XElJSVq/fr2mTp3q0K9bt246dOiQvvzyS6fnHQAAVB7E4cThZ0McThwOVGoGADzUwIEDTVhYWJmf8PBw07NnT/P+++9bbX/++WcTHh5uli5d6rCNefPmmcjISJOTk2MyMzNNWFiY2bBhg1V/9OhRM336dJOammqMMWbBggUmLCzMvPPOO1abvLw8065dO/PQQw8ZY4yZNWuWad68uUlPT3fY1+DBg027du1MSUmJMcaYsLAwc9dddzm0mTBhgomOjjbGGJfG88gjj5ibb77ZHDt2zGpz5MgRExsba2bOnHnOc3fqOJyZO3euiYyMdDiOgoICc+utt5oHH3zQGGNMWlqaCQsLM2PGjHHoe88995jExERjjDH79+834eHhZsWKFQ5thg4dasLCwkxaWto5xxQWFmbuuOMOh7IxY8aYG2+88ZzjPnbsmImIiDArV650KP/Tn/5kevfubUpLS62yd99913Tp0sVkZWWZtWvXmrCwMPP6669b9SdOnDDNmzc306dPN8YYs3v3bnPXXXcZu93usO3hw4ebrl27Wp87duxoOnfu7NDms88+M3fffbfDXBljTGJiohk6dKgxxpiffvrJhIWFmVWrVjm0GTlypPnrX/9qjDFm/PjxpmPHjlbdtm3bTFhYmHn33XfLnKd27dqZoqKic45p0qRJpmvXrtZ30hhj1q9fb15++WVzphtvvNHMmjWrTDkAAKh8iMOJw8+GOPw/54k4HKiceNkoAI/WvHlzPfnkk5JOPsY5f/58FRUVaf78+br++uutdtu3b5cxRp06dVJxcbFV3qlTJy1ZskQpKSm69dZb1bRpU02aNEnbtm1TfHy8OnTooIkTJzrsMyAgQImJidbnwMBAdejQQVu3bpUkffnll4qJiVGDBg0c+vXq1UsTJ07Uzz//rKZNm0qSoqOjHdrUq1dPeXl5kqTg4ODzjmf79u2Ki4tTYGCgdVw1atRQ69at9fnnn1/QOT3liy++UEREhEJCQqxt+/n5qUOHDmXu6DnbcRw4cECStGPHDhlj1K1bN4c2iYmJ2rZt23nHERsb6/C5YcOGZR4DPd1vv/2mkpISNWzY0CrLz8/X7t279eCDDzo8Zty9e3d1797doX/r1q2t/w8KClJwcLC1v4iICL3++usqLS3Vr7/+qv3792vv3r36+eefHb5Xp9qeLj4+XvHx8SoqKtLevXu1f/9+paamKicnR7Vq1ZIkpaSkSJK6dOni0Pf0R2zP9MUXX8hmsykhIaHMd3vDhg3as2ePNZYzx9S2bVutXr1affr0UefOnZWQkKCePXs6nKNTQkNDlZ6efs5xAACAyoU4nDj8TMThJxGHA5UXiXQAHu2KK65QZGSk9TkqKkq9evXS0KFD9fbbb6t27dqSpNzcXElSjx49zrqdQ4cOyWazacWKFVqyZIk2b96sdevWqUqVKurcubOefPJJXXXVVZJOBtYBAY5/HuvUqWPt4/fff9e1115bZh/BwcGS5BB8BgUFObTx8/OTMUaSXBpPbm6u3nvvvTLr8Emyjv1C5ebmav/+/WrevPlZ609daJzvOHJyciSdPEenO/PzuVSvXv2c2z6bY8eOlen3+++/yxjj0j6dHYskrVy5Us8//7xyc3MVHBysFi1aKCgoyNrvucZ96tHh1157TSdOnFD9+vXVsmVLVatWzWpz6jvk6rk51ccYo1atWp21PjMz0wrczxxT9+7dVVpaqtdff12LFy+2Hn8eM2ZMmQuboKCgMms2AgCAyos4nDj8TMThjojDgcqHRDoArxIcHKwnnnhCo0eP1rRp0zRnzhxJUs2aNSWdfKHLFVdcUaZfaGioJCkkJERTpkzR5MmT9eOPP+qDDz7QCy+8oKuvvlqTJ0+W9J8g63TZ2dlW0HXVVVcpKyurTJtTZVdffbXLx3O+8Vx55ZW6+eabde+995bpe+ZFhruuvPJKxcXFady4cWetP33dw/Mdg3TyHJ06z9J/AvtL7dT5Pf1CqUaNGrLZbGX2WVBQoO3btysqKsqlbW/cuFEzZ87U2LFj1adPH+siafTo0frXv/7ltO+yZcu0atUqPfnkk+rSpYuuvPJKSbLWpJT+8z3NyclRvXr1rPJ9+/YpNze3zF1B0sl5ql69usP6jqe77rrrnI4rMTFRiYmJOnbsmLZt26YXXnhBY8eOVWxsrDV30snzefr8AQAAnI44/D+Iw4nDJeJwoDLiZaMAvE63bt3Uvn17bdq0yXopy6nHBI8cOaLIyEjrJycnR88995xyc3P1zTff6Oabb9a3334rm82miIgIPfzwwwoLC1NGRoa1/fz8fH322WcOn7du3aqbbrpJknTjjTfqm2++sR6pPGXDhg2qW7fueQOqU1wZT1xcnPbu3auIiAjrmFq0aKFVq1Zp8+bNF34S/73tX375Rf/1X//lcM7Wr1+vt956S/7+/i5tJzY2Vv7+/mXG8+GHHzp89vO7NP/khISEyN/fXwcPHrTKrrjiCkVERFgvKzpl69atGjZsmDIzM13adkpKimrWrKm//OUvVvB+/PhxpaSkqLS09Lx9mzZtqr59+1rB+6FDh5Sammr1PRWgf/zxxw59n332WU2bNk1S2fMUFxenEydOyBjjME+pqalKTk4u86jr6R566CElJSVJOnkh8D//8z8aMWKEiouLHc6JMUaHDh0q85g0AADA6YjDicOJw4nDgcqMO9IBeKXHHntMvXr10tNPP6133nlH4eHh6tWrlyZNmqQDBw6oRYsW+uWXXzRv3jw1bNhQjRs3VnFxsQIDAzVu3Dg9+OCDCg4O1ueff64ffvhBgwYNctj+xIkT9dBDD6lOnTpavny5Tpw4oQceeECSdO+992rDhg0aMmSIRo4cqVq1amndunXavn27pk+f7nKgesMNN5x3PCNGjFD//v01fPhw3XXXXapWrZpWr16tjz76SAsWLDjvPlatWlWmrGbNmurTp4+GDBmi9evXa8iQIRo6dKiuvvpqvffee/rb3/5WZr1KZ6699lr17dtXc+fOVVFRkZo1a6bNmzdbwfSp83HqLpBNmzYpKirqrI/luqJ69epq1aqVUlJSNGTIEKt81KhReuCBB/TII4+od+/eys7O1ty5c9W5c2eFhYXpu+++O++2W7ZsqTfeeEMzZ85Ux44dlZmZqeXLlys7O9t65NhZ38WLF2vZsmWKjo7W/v37tXTpUhUWFlqP5zZr1kzdunXT7NmzlZ+fr4iICG3dulVbtmzRokWLrPOUnZ2tTz/9VBEREUpISNCNN96oESNGaMSIEWrSpIm+/fZbLViwQO3bt3f6aHHbtm01efJkPfPMM+rQoYOOHj2qRYsWqXHjxmrWrJnVLjU1VceOHVP79u3Pe44AAEDlRhxOHE4cThwOVFYk0gF4peuvv1733HOPVqxYoTfeeEMDBw7UjBkztHTpUr355ps6ePCg6tSpo+7du+uhhx6Sv7+//P39tWLFCs2ZM0fTpk3T0aNH1bhxYz311FPq06ePw/anTJmi6dOnKycnR61atdIbb7xh3eFSt25dvfHGG5ozZ46efvppK2hdvHixbr31VpePoVq1aucdT7NmzfTaa69p3rx5GjdunIwxCgsLU3Jyskv7mjFjRpmyRo0aqU+fPgoJCdGbb76pOXPmaMqUKSooKFDjxo01bdo0h8cgXTFp0iRVr15dK1as0B9//KGbbrpJDzzwgJKTk631Art06aL169drwoQJ6tevn6ZMmeLWPk7XtWtXLVy4UAUFBdbahx07dtTzzz+vRYsWKSkpSbVr11bPnj314IMPurzdP/3pT0pPT9fatWv1+uuvKyQkRAkJCRowYIAmTZqkffv2qUmTJmftO3z4cB05ckQvv/yykpOTVb9+fd1+++2y2WxaunSpjh49qpo1a2r27NlatGiRXnrpJR05ckRNmjTRggUL1LlzZ0lSnz599OmnnyopKUmjRo3SsGHDtGzZMj333HNaunSpDh8+rJCQEN17773WXS7n0r9/fxUVFenNN9/U66+/rsDAQN10000aO3asqlSpYrXbunWr6tate871HwEAAE4hDicOJw4nDgcqK5tx9iYJAKhkFi5cqEWLFumnn36q6KF4jdzcXG3dulXt27d3WJfymWee0dtvv60dO3Zc8n3m5eWpc+fOGjt2rHr37n3Jt1+ZGGPUtWtXDRgwwOHOIgAAgPJEHO4+4nDvRhwOeB/uSAcAXJSgoCBNmzZNERERGjx4sKpXr66dO3fq1Vdf1fDhwy/bPh988EEtX75cPXv2dHkdSZT14YcfqqSkRP3796/ooQAAAMANxOHejTgc8D68bBQAcFGqVaumVatWqVq1apowYYL+93//Vxs3btT48ePP+8jjxejfv7/q1aunNWvWXLZ9+LrCwkLNnTtXs2bNUmBgYEUPBwAAAG4gDvdexOGAd2JpFwAAAAAAAAAAnOCOdAAAAAAAAAAAnCCRDgAAAAAAAACAEyTSAQAAAAAAAABwgkQ6AAAAAAAAAABOkEgHAAAAAAAAAMAJEukAAAAAAAAAADhBIh0AAAAAAAAAACdIpAMAAAAAAAAA4ASJdAAAAAAAAAAAnPh/S+Xnt9mS9UAAAAAASUVORK5CYII=",
      "text/plain": [
       "<Figure size 1500x600 with 2 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Chosen responses - Mean length: 1447.0, Median length: 1371.5\n",
      "Rejected responses - Mean length: 1468.3, Median length: 1355.5\n",
      "Total chosen responses: 9998\n",
      "Total rejected responses: 9998\n"
     ]
    }
   ],
   "source": [
    "# Set the plotting style\n",
    "plt.style.use('seaborn-v0_8')\n",
    "sns.set_palette(\"husl\")\n",
    "\n",
    "# Use two side-by-side subplots\n",
    "fig, (ax1, ax2) = plt.subplots(1, 2, figsize=(15, 6))\n",
    "\n",
    "# Plot the distribution of chosen response lengths\n",
    "chosen_lengths = list(chosen_len_cnt.keys())\n",
    "chosen_counts = list(chosen_len_cnt.values())\n",
    "\n",
    "ax1.hist(chosen_lengths, weights=chosen_counts, bins=50, alpha=0.7, color='skyblue', edgecolor='black')\n",
    "ax1.set_title('Distribution of Chosen Response Lengths', fontsize=14, fontweight='bold')\n",
    "ax1.set_xlabel('Response Length (characters)', fontsize=12)\n",
    "ax1.set_ylabel('Frequency', fontsize=12)\n",
    "ax1.grid(True, alpha=0.3)\n",
    "\n",
    "# Plot the distribution of rejected response lengths\n",
    "rejected_lengths = list(rejected_len_cnt.keys())\n",
    "rejected_counts = list(rejected_len_cnt.values())\n",
    "\n",
    "ax2.hist(rejected_lengths, weights=rejected_counts, bins=50, alpha=0.7, color='lightcoral', edgecolor='black')\n",
    "ax2.set_title('Distribution of Rejected Response Lengths', fontsize=14, fontweight='bold')\n",
    "ax2.set_xlabel('Response Length (characters)', fontsize=12)\n",
    "ax2.set_ylabel('Frequency', fontsize=12)\n",
    "ax2.grid(True, alpha=0.3)\n",
    "\n",
    "plt.tight_layout()\n",
    "plt.show()\n",
    "\n",
    "# Use pandas Series to directly calculate statistics\n",
    "chosen_series = pd.Series([length for length, count in chosen_len_cnt.items() for _ in range(count)])\n",
    "rejected_series = pd.Series([length for length, count in rejected_len_cnt.items() for _ in range(count)])\n",
    "\n",
    "# Display statistical information\n",
    "print(f\"Chosen responses - Mean length: {chosen_series.mean():.1f}, Median length: {chosen_series.median():.1f}\")\n",
    "print(f\"Rejected responses - Mean length: {rejected_series.mean():.1f}, Median length: {rejected_series.median():.1f}\")\n",
    "print(f\"Total chosen responses: {len(chosen_series)}\")\n",
    "print(f\"Total rejected responses: {len(rejected_series)}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "40f3cbe1",
   "metadata": {},
   "source": [
    "### 3.3 Evaluating Model Baseline Capabilities\n",
    "Define evaluation criteria."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "id": "eebe9a3b",
   "metadata": {},
   "outputs": [],
   "source": [
    "def evaluate_dpo_model(testset_path):\n",
    "    output_lens = []\n",
    "    with open(testset_path, \"r\") as fin:\n",
    "        for line in fin:\n",
    "            data = json.loads(line.strip())\n",
    "            if data[-1].get(\"role\", \"\") == \"bot\":\n",
    "                output_lens.append(len(data[-1].get(\"content\", \"\")))\n",
    "    return sum(output_lens) / len(output_lens), sorted(output_lens)[len(output_lens) // 2]"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "0a3687fe",
   "metadata": {},
   "source": [
    "Construct the test set. We randomly sample 200 entries from the raw data and convert them to OpenAI API format as the test set."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "id": "c3d68264",
   "metadata": {},
   "outputs": [],
   "source": [
    "def ernie_data_to_openai_api_format(data, keep_system: bool = True):\n",
    "    openai_data = []\n",
    "    if keep_system and data.get(\"system\", \"\"):\n",
    "        openai_data.append({\"role\": \"system\", \"content\": data.get(\"system\", \"\")})\n",
    "    if len(data[\"src\"]) == 1:\n",
    "        openai_data.append({\"role\": \"user\", \"content\": data[\"src\"][0]})\n",
    "    else:\n",
    "        for i in range(len(data[\"src\"]) - 1):\n",
    "            openai_data.append({\"role\": \"user\", \"content\": data[\"src\"][i]})\n",
    "            openai_data.append({\"role\": \"assistant\", \"content\": data[\"tgt\"][i]})\n",
    "        openai_data.append({\"role\": \"user\", \"content\": data[\"src\"][-1]})\n",
    "    return openai_data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "id": "00ab689b",
   "metadata": {},
   "outputs": [],
   "source": [
    "testset_path = \"../data/dpo_testset.jsonl\"\n",
    "random.seed(RANDOM_SEED)\n",
    "\n",
    "with open(ernie_data_path, \"r\") as fin:\n",
    "    data_list = [json.loads(line) for line in fin]\n",
    "    test_data = random.sample(data_list, 200)\n",
    "    # We noticed that some system prompts in the raw data require the model to deliberately output long text (e.g., 'Must generate a detailed and lengthy response'),\n",
    "    # which may affect the model's performance, so we do not retain the system prompt\n",
    "    test_data = [ernie_data_to_openai_api_format(data, keep_system=False) for data in test_data]\n",
    "\n",
    "with open(testset_path, \"w\", encoding=\"utf-8\") as fout:\n",
    "    for item in test_data:\n",
    "        fout.write(json.dumps(item, ensure_ascii=False) + \"\\n\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "262dbe2a",
   "metadata": {},
   "source": [
    "Use the original model to infer the test set. Currently, the WebUI does not support inferring test sets, so we use the inference scripts from the ERNIEKit codebase for this operation.\n",
    "- **Script**: `tools/inference/scripts/infer.sh`\n",
    "- **Set key hyperparameters**:\n",
    "  - `top_p=0.7`\n",
    "  - `temperature=0.7`\n",
    "  - `max_seq_len=128000` (ERNIE-4.5-0.3B supports dynamic 128K context length)\n",
    "  - `max_dec_len=8192` (since this task focuses on the model's output length, avoid actively truncating the model's output)\n",
    "  - `weight_quantize_algo` Remove this line if quantization was not used during training\n",
    "\n",
    "> Inference process omitted"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a6c3137d",
   "metadata": {},
   "source": [
    "Calculate evaluation metrics."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "5558bb29",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Get inference results\n",
    "dpo_infer_path = \"../data/infer/dpo_03b_exp0.jsonl\"\n",
    "raw_model_output_len_mean, raw_model_output_len_median = evaluate_dpo_model(dpo_infer_path)\n",
    "print(\"Original Model\")\n",
    "print(f\"Output Length Mean: {raw_model_output_len_mean:.1f}, Median: {raw_model_output_len_median}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "5a67ae5d",
   "metadata": {},
   "source": [
    "### 3.4 Making Training Set\n",
    "How to construct it most effectively? Generally, the `rejected` in the training set is the actual output of the model to be optimized, while `chosen` is the optimized result based on this output. Here, we directly use the `rejected` and `chosen` from the open-source dataset. It’s not hard to think of two basic approaches to constructing the training set: one is where `rejected` is close to the output length of the original model on the test set and `rejected` is significantly longer than `chosen`; the other does not consider the output length of the original model on the test set, as long as `rejected` is significantly longer than `chosen`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "effcd235",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Training Set 1: rejected length close to the original model\n",
    "with open(ernie_data_path, \"r\") as fin:\n",
    "    data_list = [json.loads(line) for line in fin]\n",
    "    trainset_1 = []\n",
    "    for data in data_list[201:]:\n",
    "        chosen = data[\"response\"][0]\n",
    "        rejected = data[\"response\"][1]\n",
    "        # Use the mean as a reference, allowing a certain range of deviation\n",
    "        if (\n",
    "            abs(len(rejected) - raw_model_output_len_mean) < raw_model_output_len_mean * 0.4\n",
    "            and len(rejected) > len(chosen) * 1.2\n",
    "        ):\n",
    "            trainset_1.append(data)\n",
    "\n",
    "print(f\"Training Set 1 sample size: {len(trainset_1)}\")\n",
    "print(\n",
    "    f\"Average length of rejected in Training Set 1: {sum(len(d['response'][1]) for d in trainset_1) / len(trainset_1):.1f}\"\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 37,
   "id": "cc5888a0",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Training Set 2: rejected significantly longer than chosen\n",
    "with open(ernie_data_path, \"r\") as fin:\n",
    "    data_list = [json.loads(line) for line in fin]\n",
    "    trainset_2 = []\n",
    "    for data in data_list[201:]:\n",
    "        chosen = data[\"response\"][0]\n",
    "        rejected = data[\"response\"][1]\n",
    "        if len(rejected) > len(chosen) * 1.2:\n",
    "            trainset_2.append(data)\n",
    "\n",
    "print(f\"Training Set 2 sample size: {len(trainset_2)}\")\n",
    "print(\n",
    "    f\"Average length of rejected in Training Set 2: {sum(len(d['response'][1]) for d in trainset_2) / len(trainset_2):.1f}\"\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "64b74cba",
   "metadata": {},
   "source": [
    "Control variables to ensure the sample sizes of the two training sets are the same."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 38,
   "id": "cac5b121",
   "metadata": {},
   "outputs": [],
   "source": [
    "random.seed(RANDOM_SEED)\n",
    "trainset_2 = random.sample(trainset_2, len(trainset_1))\n",
    "\n",
    "print(f\"Training Set 2 sample size: {len(trainset_2)}\")\n",
    "print(\n",
    "    f\"Average length of rejected in Training Set 2: {sum(len(d['response'][1]) for d in trainset_2) / len(trainset_2):.1f}\"\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6e54f7d1",
   "metadata": {},
   "source": [
    "Export the training sets."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 39,
   "id": "4772b226",
   "metadata": {},
   "outputs": [],
   "source": [
    "trainset_1_path = \"../data/dpo_trainset_exp1.jsonl\"\n",
    "trainset_2_path = \"../data/dpo_trainset_exp2.jsonl\"\n",
    "\n",
    "# We noticed that some system prompts in the raw data require the model to deliberately output long text (e.g., 'Must generate a detailed and lengthy response'),\n",
    "# which may affect the model's performance, so we do not retain the system prompt\n",
    "with open(trainset_1_path, \"w\") as fout:\n",
    "    for data in trainset_1:\n",
    "        del data[\"system\"]\n",
    "        fout.write(json.dumps(data, ensure_ascii=False) + \"\\n\")\n",
    "\n",
    "with open(trainset_2_path, \"w\") as fout:\n",
    "    for data in trainset_2:\n",
    "        del data[\"system\"]\n",
    "        fout.write(json.dumps(data, ensure_ascii=False) + \"\\n\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "0c0f1a02",
   "metadata": {},
   "source": [
    "# 4 Training\n",
    "### 4.1 Choosing the Base Model\n",
    "Considering the convenience of demonstration and the complexity of the task, we choose the smaller-sized ERNIE-4.5-0.3B model.\n",
    "\n",
    "### 4.2 Designing Training Parameters\n",
    "Designing training parameters relies heavily on experience.\n",
    "\n",
    "### 4.3 Training with WebUI\n",
    "1. Start the WebUI in the root directory of the ERNIEKit project: `erniekit webui`  \n",
    "2. Configure the model path and export directory  \n",
    "\n",
    "![dpo_set_path](https://raw.githubusercontent.com/wiki/Minghao2812/ERNIE/img/dpo_set_path_en.png)\n",
    "\n",
    "3. Set full-parameter/LoRA fine-tuning, numerical precision, etc.  \n",
    "\n",
    "![dpo_lora_set](https://raw.githubusercontent.com/wiki/Minghao2812/ERNIE/img/dpo_set_lora_en.png)\n",
    "\n",
    "4. Set the fine-tuning mode (SFT/DPO) and configure training parameters  \n",
    "\n",
    "![dpo_train_params](https://raw.githubusercontent.com/wiki/Minghao2812/ERNIE/img/dpo_train_params_en.png)\n",
    "\n",
    "5. Configure the training set and validation set\n",
    "\n",
    "<div style=\"display: flex; justify-content: space-around;\">\n",
    "    <img src=\"https://raw.githubusercontent.com/wiki/Minghao2812/ERNIE/img/dpo_trainset_en.png\" alt=\"dpo_trainset\" style=\"width: 49%;\">\n",
    "    <img src=\"https://raw.githubusercontent.com/wiki/Minghao2812/ERNIE/img/dpo_validset_en.png\" alt=\"dpo_validset\" style=\"width: 49%;\">\n",
    "</div>\n",
    "\n",
    "### 4.4 Viewing Training Logs and Loss Curves\n",
    "Training log path: `${your_model_dir}/paddle_dist_log/workerlog.0`. You can also run the following command to view the loss curve:\n",
    "```bash\n",
    "visualdl --logdir ${your_model_dir}/vdl_log --host 0.0.0.0\n",
    "```\n",
    "\n",
    "### 4.5 Merging Model Weights (Only Required for LoRA Fine-Tuning)\n",
    "You can conveniently merge model weights in the 'Evaluation' mode of the WebUI   \n",
    "\n",
    "![dpo_merge_lora](https://raw.githubusercontent.com/wiki/Minghao2812/ERNIE/img/dpo_merge_lora_en.png)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "62dcd43b",
   "metadata": {},
   "source": [
    "# 5 Performance Evaluation"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6d53d017",
   "metadata": {},
   "source": [
    "### 5.1 Calculating Evaluation Metrics\n",
    "The evaluation criteria have been defined above, and the method for calculating evaluation metrics has been demonstrated. We will not repeat the process here, but instead present the training configurations and evaluation metrics of several experiments for readers' reference.\n",
    "\n",
    "| Experiment No. | Training Set | Training Parameters | ↓ Output Length Mean | ↓ Output Length Median |\n",
    "| --- | --- | --- | --- | --- |\n",
    "| 0 | - | - | 888.8 | 515 |\n",
    "| 1 | • len(rejected) close to original model<br>• len(rejected) >= 1.2 len(chosen)<br>• 614Q | max_steps=614<br>warmup_steps=50<br>global_batch_size=1 | 879.5 | 442 |\n",
    "| 2 | • len(rejected) >= 1.2 len(chosen)<br>• 614Q | max_steps=614<br>warmup_steps=50<br>global_batch_size=1 | 864.7 | 354 |"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a6639187",
   "metadata": {},
   "source": [
    "### 5.2 Analyzing Experimental Results\n",
    "It can be seen that under the same training parameters, Training Set 2 is more effective. This naturally leads to the question: if we include more training data and make the length difference between `rejected` and `chosen` more significant, would the model's performance improve further? We designed a third set of experiments, and the results are as follows:\n",
    "\n",
    "| Experiment No. | Training Set | Training Parameters | ↓ Output Length Mean | ↓ Output Length Median |\n",
    "| ---- | ---- | ---- | ---- | ---- |\n",
    "| 0 | - | - | 888.8 | 515 |\n",
    "| 1 | len(rejected) close to original model <br> len(rejected) >= 1.2 len(chosen) <br> 614Q | max_steps=614 <br> warmup_steps=50 <br> global_batch_size=1 | 879.5 | 442 |\n",
    "| 2 | len(rejected) >= 1.2 len(chosen) <br> 614Q | max_steps=614 <br> warmup_steps=50 <br> global_batch_size=1 | <ins>**864.7**</ins> | <ins>**354**</ins> |\n",
    "| 3 | len(rejected) >= 2 len(chosen) <br> 1804Q | max_steps=1804 <br> warmup_steps=50 <br> global_batch_size=1 | 1145.1 | 412 |\n",
    "\n",
    "Unfortunately, the model's performance worsened. We have the following hypotheses:  \n",
    "1. Insufficient warm-up. Based on general experience, setting warm-up to 10% of max_steps might be better.  \n",
    "2. Too few max_steps.  \n",
    "\n",
    "To address these two hypotheses, we designed a fourth set of experiments. The results are as follows:\n",
    "\n",
    "| Experiment No. | Training Set | Training Parameters | ↓ Output Length Mean | ↓ Output Length Median |\n",
    "| ---- | ---- | ---- | ---- | ---- |\n",
    "| 0 | - | - | 888.8 | 515 |\n",
    "| 1 | • len(rejected) close to original model <br> • len(rejected) >= 1.2 len(chosen) <br> • 614Q | max_steps=614 <br> warmup_steps=50 <br> global_batch_size=1 | 879.5 | 442 |\n",
    "| 2 | • len(rejected) >= 1.2 len(chosen) <br> • 614Q | max_steps=614 <br> warmup_steps=50 <br> global_batch_size=1 | 864.7 | 354 |\n",
    "| 3 | • len(rejected) >= 2 len(chosen) <br> • 1804Q | max_steps=1804 <br> warmup_steps=50 <br> global_batch_size=1 | 1145.1 | 412 |\n",
    "| 4 | • len(rejected) >= 2 len(chosen) <br> • 1804Q | max_steps=3608 <br> warmup_steps=360 <br> global_batch_size=1 | <ins>**628.1**</ins> | <ins>**331**</ins> |\n",
    "\n",
    "It can be seen that the fourth set of experiments achieved more desirable results!"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ed3926fe",
   "metadata": {},
   "source": [
    "### 5.3 Catastrophic Forgetting\n",
    "We attempted to test whether the model's performance on general tasks declined (exhibiting a 'forgetting phenomenon'). The good news is that 'forgetting' is not significant. For example:\n",
    "> Input: [{\"role\": \"user\", \"content\": \"Please remember, your name is Sponge\"}, {\"role\": \"assistant\", \"content\": \"Okay, I am Sponge\"}, {\"role\": \"user\", \"content\": \"Who are you?\"}]  \n",
    "\n",
    "> ERNIE-4.5-0.3B Original Model Output: I am Sponge, a super curious and fun guy! Is there anything you'd like to chat about?  \n",
    "> DPO Model Output: I am Sponge, a psychic warrior exploring the sponge world!"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "ar",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.18"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}
